Over the past year, the WMS has faced increasing pressure from automated web traffic, mostly generated by bots used for AI training. After a major outage last year, we implemented a temporary solution using our existing on‑premises systems.
While this helped restore service, it was not a long-term solution. On May 27, the WMS experienced another outage affecting off‑campus users. In response, we accelerated the deployment of Azure Front Door, a Microsoft cloud service that helps protect and speed up websites. We had already been preparing this rollout, and planned to deploy it in the coming weeks. Our colleagues at McGill Libraries faced similar challenges with their own systems and also transitioned to Azure Front Door.
This article explains what changed, what it means for you as a content editor, and what to expect moving forward.
What has changed for site visitors
Introducing Azure Front Door improves reliability and performance, but it also introduces a few behaviours you and your users may notice.
-
A 10-minute cache for public content
For visitors who are not logged in, pages are now cached for up to 10 minutes.
What this means:
- When you publish or update content, changes may not appear immediately to the general public
- Logged-in users will still see updates right away
- After the cache expires, the new content becomes visible automatically
Why this is important:
This caching significantly reduces the load on our servers and helps absorb traffic spikes, making outages less likely.
-
JavaScript challenge on certain pages
To verify that visitors are real users, some parts of the WMS (such as the campus search interface) may show a quick browser check to confirm the visitor is not a bot.
What users may see:
- A brief verification step when accessing certain pages
- The page loads normally after the check completes
Legitimate users should not experience issues, but users with JavaScript disabled may not be able to access these features.
-
Occasional blocking pages
Security filters may occasionally block real users by mistake.
What users may see:
- A blocking page displaying a Support ID
- A "403 Forbidden" error message (access denied)
- A page that appears unstyled or incomplete
What to do when something does not work
Because we are still stabilizing the system, your feedback is essential.
Before reporting a problem, try reproducing the issue without browser extensions (for instance using Chrome's Guest profile).
If you still see the issue, please report it as soon as possible via the WMS issue form.
Please include:
- The Support ID (if shown on the error page)
- The URL of the page
- A description of the issue
This information helps us quickly identify and update the settings that control what traffic is allowed or blocked.
What are WAF, CDN, and bot mitigation services?
Modern websites use several tools to stay secure and run effectively. At McGill, these capabilities are now provided through Azure Front Door:
- A Web Application Firewall (WAF) filters harmful or suspicious activity before it reaches the WMS.
- A Content Delivery Network (CDN) saves copies of web pages in different locations around the world so they load faster for users.
- A bot mitigation service detects and controls automated traffic – while some bots are legitimate (e.g. search engines), others, such as AI scraping or malicious crawlers, can overwhelm websites and cause outages.
Why these protections are now essential
Automated traffic on the Internet has increased significantly. Large institutional websites like McGill's are frequent targets – not necessarily for attacks in the traditional sense, but for large-scale automated data collection.
Recent reporting shows that automated traffic now exceeds human activity online, marking a significant shift in how the Internet is used. AI agents and bots now make up most of the traffic to websites, and that growth is accelerating rapidly.
On May 27, the WMS received an unusually high volume of traffic from external sources. Our previous solution could not reliably tell the difference between real users and bots, resulting in widespread access issues.
Azure Front Door provides:
- Better detection and blocking of unwanted traffic
- More detailed information to help us monitor issues
- Faster and more reliable performance thanks to a global network
Since its deployment, the WMS has been mostly stable, and we continue to adjust and improve the system.
Ongoing improvements
We are actively monitoring and improving how the system is set up in collaboration with Microsoft specialists. While the platform has been mostly stable since deployment, adjustments will continue as we:
- Reduce instances where legitimate users are blocked
- Improve how we detect and manage bots
- Make the system both secure and easy to use
Thank you for your patience
These changes are an important step toward ensuring the long-term reliability of the WMS in an evolving web landscape.
We understand that the 10-minute cache and occasional access challenges may require adjustment, but they play a crucial role in preventing outages and maintaining performance.
Your feedback is critical in helping us improve the system – please continue reporting any issues you encounter.
