Reading time:
11 min

5 Times You Should Be Monitoring The Microsoft 365 User Experience

By Nick Cavalancia, Microsoft MVP


With over 200 million users of Microsoft 365 worldwide, it’s evident that organizations are putting the productivity of their users into the hands of Microsoft’s cloud-based solution. From communications using products like Exchange Online, Teams, and Skype; to collaboration using SharePoint Online; to leveraging OneDrive for document storage and sharing; organizations shifting to Microsoft 365 rely heavily on it as the very foundation for their operational ecosystem.

And for larger organizations with many locations and thousands of users, putting all their proverbial eggs in the Microsoft 365 basket introduces a material amount of risk: if users don’t properly adopt Microsoft’s cloud service as their way of doing business, like any new technology, it’s either going to be a huge failure or, at a minimum, result in increased costs.

In essence, productivity on the platform defines whether Microsoft 365 is a success.

And yet, organizations aren’t certain how to measure productivity. Most simply focus on whether the service is available or not; but the reality is that doesn’t define whether a user can adequately use the service. What organizations are missing is the ability to measure the user experience.

Think about it: there’s a lot involved between a user request made on a local workstation and the response by Microsoft 365; local network connectivity, DNS, Internet routing, latency, authentication, etc. all can go wrong, resulting in a bad user experience. And IT is responsible, no matter where an issue lies. But by measuring the user experience – putting objective metrics to the end-to-end experience had by users – organizations can quantify productivity.

Keep in mind, it’s not just about ensuring Microsoft 365 is working properly from a user perspective. This is actually about achieving three objectives as you deploy and use Microsoft 365:

  • Improve organizational productivity
    How can you measure whether an Microsoft 365 user can be productive or not?
  • Improve service quality
    What kinds of changes to the network, to endpoints, and to the connectivity to Microsoft 365 make the quality of service better?
  • Lower the total cost of ownership
    Productivity is subjective while the TCO of Microsoft 365 is objective. How can you use one to justify the other?

Whether hybrid in approach or using the cloud as a permanent strategy, organizations assume and require certain minimum levels of acceptable performance as means to ensure productivity and profitability. So, given the material reliance upon Microsoft 365, it’s crucial that organizations measure the user experience in an effort to achieve the three aforementioned objectives.

But the question first needs to be answered, “How should you go about measuring the user experience?”

Measuring the User Experience

When you boil it all down, users need to be productive when using Microsoft 365. But there are a lot of factors involved that can influence the user experience that make it a challenge to determine, if there’s a problem, what the root issue is. These factors include:

  • User Hardware
  • WiFi/Network Connectivity
  • Security Solutions (Firewalls, SSL decryption, etc.)
  • Internet Access & Routing Authentication (e.g., Azure AD Sync, ADFS)
  • Hybrid Components (e.g., Exchange Server, Edge)
  • Microsoft Infrastructure
  • Network Latency

With so many possible sources of issues around performance, response time, timeout errors, and even just user perception, how should you measure whether a user is achieving an acceptable experience with Microsoft 365 not?

It needs to be an objective means measurement; one that eliminates a lot of the noise that can occur when two factors can proverbially “point” at each other. Microsoft offers a health status for all its services at which provides the status of all services that are a part of your subscription, as well as any known incidents and advisories. But this set of, what is effectively, server-side metrics only tells one part of the user experience tale, as it only covers everything from within the “walls” of Microsoft 365.

Client-side monitoring has the same issue; because we’re talking about monitoring users interacting with a SaaS application, the measurement must represent the entire path from the client to the service and back.

Synthetic Monitoring

Gartner recently released a Technical Professional Article entitled Use Synthetic Monitoring to Enhance User Experience for Hosted and SaaS Applications where they introduce the concept of synthetic transactions as a way to monitor performance. In the case of Microsoft 365, real user interaction with Microsoft 365 is mimicked – including authentication, file upload/download, and use of individual services. These transactions provide organizations with visibility into measured experiential metrics that can be used to identify performance issues and root causes of problems. Because the transactions occur within the very same paths that users themselves take, they offer a real-world view into what users are experiencing well before users have “had enough” and inform IT of an issue.

Other forms of monitoring Microsoft 365 performance lack the contextual detail synthetic monitoring can offer because they typically are relying on bigger picture monitoring metrics (such as the previously mentioned Microsoft 365 health status) that isn’t helpful with, say, attempting to isolate whether it’s your WiFi that is causing a performance issue.

There are a few points in the journey to Microsoft 365 where monitoring the user experience via synthetic monitoring can be useful. The remainder of this paper will cover five specific stages within an Microsoft 365 deployment that need to include assessing the user experience to ensure those needed levels of performance. It will discuss what can impact the user experience in each of the five stages, and how monitoring the user experience should be done to lower the total cost of Microsoft 365 ownership while improving organizational productivity.

Stage 1: Architecting

Smaller network environments simply have a connection to the Internet and, therefore, Microsoft 365. But as an organization grows, adds on locations, acquires new companies, and seeks to ensure the entire network falls subject to centralized security and policies, the path from endpoint to Microsoft 365 is often very complicated.

And because Microsoft knows this, they’ve published guidance on how they do and don’t want you architecting your connectivity to Microsoft 365. But, as organizations often do, it becomes a simple matter of testing out connectivity and, if it seems fast enough, it’s good enough as is. But that was a test of one or two users; what happens when there will be hundreds or thousands of users utilizing the architected connectivity?

While Microsoft would simply have you follow their best practices, test out the user experience with your intended proxy servers, VPNs, firewalls, etc. consider performing tests on a site-bysite basis to ensure consistent performance. Also, if possible, consider simulating a workload representative of the planned workforce to both determine what the experience will be like, and what impact removing any parts of the current network path would have on the speed of interaction between the user and Microsoft 365. By testing the user experience using synthetic transactions before any actual deployment occurs, IT can prevent unexpected issues with a high confidence that they will occur without making architectural changes.

Stage 2: Deploying

When making the move from on-premises Microsoft solutions to Microsoft 365, users themselves don’t exactly know what to expect. For example, once you moved their mailbox into Exchange Online, should they be seeing a response time equivalent to what they did when email was on-premises? Same goes for any other service moved into Microsoft 365. And so, because users don’t know whether what they are experiencing is “normal” or not, many do not complain; instead they may simply decide to work around any slowdowns in their work process and use some other solution.

Adoption of a new solution can make or break its success in the organization. So, it’s imperative that IT has an understanding of whether users that have recently made the switch to Microsoft 365 are enjoying equivalent levels of perceived performance to their previously on-premises service counterparts (such as utilizing Exchange on-premises to Exchange Online). Monitoring user experience at this stage helps to ensure adoption and provide a feedback loop for IT to be aware of any performance issues users are experiencing – in this case – well-before the user ever knows what they are experiencing isn’t “normal.”

At the same time, IT has a plan of how a deployment should go (“should” being the operative phrase…). Testing the user experience for each phase of a deployment ensures the plan’s assumed performance is met, which helps ensure the projected costs remain under budget. It also allows IT to ensure a continuous level of service quality is maintained throughout the deployment.

Stage 3: Monitoring

As with rolling out any new service the organization depends on, on-going monitoring is necessary. It’s your way of ensuring service delivery, service quality, spotting issues before they impact users, and is IT’s source of context and detail used to better understand the problem and formulate a potential solution.

But unlike monitoring operating systems, network traffic, applications, and services, this isn’t about whether something is simply up or down; this is about putting objective metrics in place to represent what is normally a subjective “good” or “bad” user experience.

Ongoing monitoring of Microsoft 365 using synthetic transactions helps IT organizations in a number of ways:

  • Understanding the performance impact any network issues have on those working with Microsoft 365
  • Providing tangible service quality data to Microsoft, should they need to be involved in resolving an issue
  • Calculating the performance gains and resulting ROI when making improvements that positively impact Microsoft 365 users

Monitoring Microsoft 365 shouldn’t just be a “red light/green light” mentality; with synthetic transaction-based monitoring of Microsoft 365, IT organizations can lower the overall cost of their investment through improved usage and lowered support costs.

Stage 4: Troubleshooting

Let’s suppose a user calls the helpdesk and says “Outlook is slow”. It’s merely perception, is a vague complaint at best, and provides no directional guidance on where helpdesk teams are to start. On top of that, the number of users experiencing the same issue that didn’t call the helpdesk is certainly some multiple of the count of those that did. Once it’s determined that the client itself isn’t the problem, how is IT supposed to solve the verified performance issue? Microsoft support isn’t going to go much farther than “our services are working,” and so IT is left with little more than knowing each side of the interaction is functioning.

So, how can IT best troubleshoot any problems?

Using synthetic transactions to simulate the user experience can help organizations look beyond basic network-related issues and identify the root cause of the issue. By testing the user experience using different conditions such as location, use of security solutions or network services that cause indirect routes to Microsoft 365, and authentication methods, organizations can work to isolate where exactly the issue lies.

For example, if half of your locations are experiencing slow Microsoft 365 performance, and believe a proxy server to be the culprit, simulating a user that bypasses the proxy can produce connectivity metrics that can definitively conclude it’s the proxy that’s the root cause.

In general, IT troubleshooting is about isolating each part of the equation and working to isolate the problem. Without synthetic transaction-based monitoring, it’d be nearly impossible to get any more granular than each end of the Microsoft 365 interaction (that is, the client and the service) saying “it’s working over here on my end”.

Stage 5: Optimizing

As the organization changes over time – e.g., new locations, acquisitions, new security and compliance initiatives, and improvements in network connectivity – IT needs to understand exactly how these changes will impact Microsoft 365 use by testing the impact on performance should changes to the network and network services occur.

At the same time, many organizations have moved to the cloud before upgrading any of their local network that was initially designed for local service delivery in mind. The resulting slower cloud performance – because of the high reliance on Microsoft 365 to be productive – may finally cause an organization to work to adopt the network connectivity best practices outlined by Microsoft.

Whether the changes are a natural part of an organization’s evolution, or a concerted choice to adopt standard connectivity principles, monitoring the user experience via synthetic transactions helps ensure that each change intended to further optimize the connection between the user and Microsoft 365 is, indeed, having a positive impact.

Getting the Most from Microsoft 365 by Monitoring the User Experience

Organizations that heavily depend on Microsoft 365 simply can’t afford to not have visibility into exactly what users are experiencing. By monitoring the Microsoft 3655 user experience using synthetic monitoring, organizations place objective metrics onto what is normally subjective perception, providing IT with needed insight. This necessary monitoring uniquely allows organizations to determine how to design proper Microsoft 365 connectivity, verify successful deployments, ensure service delivery, identify problem root causes, and improve user connectivity and productivity.

About the Author

Nick Cavalancia is a Microsoft Cloud and Datacenter MVP and has over 25 years of IT experience dealing with the architecture, implementation and training of Microsoft technologies to enterprise customers.

Nick has attained industry certifications including MCSE, MCT, MCNE, and MCNI. He has authored, co-authored and contributed to over a dozen books on Windows, Active Directory, Exchange and other Microsoft technologies and has spoken at many technical conferences on a wide variety of topics.

See how CoreView can help you with this

Learn more about securing and optimizing your M365 and other SaaS applications.