Google's fast AMP framework and Core Web Vitals, pass or fail?

I stopped supporting AMP within my own boilerplate CMS, as I achieved better pagespeed results without AMP, because of the increased freedom. Having worked on both sides -optimizing pagespeed with and without AMP-, the experience might give me some good insights while assessing AMP and Core Web Vitals.

Why AMP was necessary

AMP was only needed because most websites and platforms forgot to keep an eye on performance. Moreover, developers weren't doing a great job either. And on top of that, most publisher websites got tanked with third party scripts. This all lead to poor pagespeed and bad user experience.

Google created AMP and it got traction quite fast. Maybe also because publishers got afraid that other publishers would benefit from having AMP pages . FOMO might have played a role here.

Double the work

Remember the pre-responsive webdesign era? We had mobile-only environments, forcing us to maintain two seperate versions of a website and its templates. We basically went back to that situation, as articles were published in two versions:

• the regular version, obviously also used for desktop;
• the AMP equivalent, used for traffic coming from Google or social media when using a mobile device.

Being restricted and having specific requirements within the AMP framework, this could take up quite some time, depending on your type of pages.

Google actually cheated with AMP

Not sure if you could call it cheating, as it also is a best practice: Google would prefetch AMP libraries and results within the search engine results pages. However, Google did not do this for non-AMP pages. As a result, pages published in AMP would always have a head-start and would feel way faster.

But that's not all. It even impacted ads:

Google also [redacted] of non-AMP ads by giving them artificial one second delays

Brad Neuberg via Twitter

That wasn't a great look and surprised me to be honest.

Core Web Vitals replacing AMP

But we are here for an AMP Core Web Vitals assessment. The most important update might be that you don't need AMP anymore to let your pages become part of Google Top Stories feature.

ranking criteria for the Top Stories feature in Search on mobile, and remove the AMP requirement from Top Stories eligibility

developers.google.com

This is big step forwards, as website and webshop owners in general are now encouraged to also start addressing the pagespeed and overall user experience of the regular website and page types.

AMP and Core Web Vitals, does it pass?

They finally do, but only recently as AMP now addressed their last challenge: the First Input Delay metric. Let's look back.

AMP's CrUX data of their Core Web Vitals metrics

Up to November 2020, Google's very own amp.dev environment did not pass their newly introduced Core Web Vitals, as they did not pass the First Input Delay metric.

This is what we see within this screenshot:

• real user experience data on December 13th, 2020;
• on mobile devices for the homepage as well as overall website experience;
• the 75th percentile LCP of the homepage is 3 seconds, while the overall LCP 75th percentile is at 2.1 seconds;
• but even today, these LCP scores as well as their user experience distributions (colored bars) are the same;
• the First Input Delay improved by a lot: from 231ms to 48ms within origin summary data.

Correlation between Core Web Vitals and AMP

According to AMP, there is a correlation between Core Web Vitals and AMP. And to be honest: there is, as most aspects within AMP really are best practices. But to improve pagespeed, actually all of them can be implemented within non-AMP architecture as well.

As a coincidence, the AMP blogpost forgot to mention the third metric: First Input Delay. Maybe because they knew they had difficulties with AMP passing Core Web Vitals before.

The official AMP site now passes Core Web Vitals assessment

However, Google did announce improvements within the AMP framework in November 2020, to be able to improve both LCP and FID metrics. This should improve metrics of all AMP websites, as the changes are made within the JavaScript libraries as hosted by AMP themselves.

We are looking to ship modern JS to browsers to continue to reduce the AMP payload

blog.amp.dev

Despite the article of November 2020, LCP did not improve. The First Input Delay metric did though, by a lot.

While this is mobile data, this is also the 75th percentile of all visitors, regardless of the page they were visiting. In other words: these are the origin summary Core Web Vitals data for the official AMP.dev website.

But thanks to open CrUX data, we are able to determine improvements in specific countries as well. Apparently, the following countries are not using the modern JS yet as a result of different circumstances:

• Brazil, the FID went from 275ms in November to 125ms in December;
• Italy, 225ms to 100ms;
• Mexico, 250ms to 100ms;
• Thailand, from 225ms to 100ms;
• Poland, from 225ms in October to 175ms in December (no mobile November data for Poland).

This might mean that still quite some mobile users within these countries typically are on older phones and/or not updating their operating systems and browsers as often as users from other countries.

JavaScript is the reason for AMP's poor Core Web Vitals

AMP will load JavaScript asynchronously. That's a good thing, as for example First Contentful Paint will happen very fast. Especially as CSS is inlined, preventing render blocking resources.

However, this best practice is likely to introduce bottlenecks. As user engagement can already happen, users might start to interact with webpages while JavaScript is still being executed. Remember the main thread blocking demo as part of my chatbot pagespeed impact visualizer? This is basically what happened for more than 25% of mobile amp.dev visitors.

Looking at public data one more time, we can see that the Load Event experience did not improve, despite improved First Input Delay data. Load Event data even regressed.

According to Google themselves, the Load Event should be within the 2.5 seconds threshold for at least 75% of users, just like the LCP metric. Although the Load Event is not part of Core Web Vitals assessment (yet?), they are not passing this specific metric.

Other mobile insights:

• The Netherlands has the best mobile 75th percentile Load Event: 1.8 seconds;
• Countries such as Argentina (5.1 seconds), Brazil (5.0 seconds), Pakistan (5.4 seconds) have the worst Load Event experience.

AMP Core Web Vitals conclusion

AMP provides you with a good architecture or boilerplate to achieve optimal experience for your articles, product pages or other pages. However:

• you should consider both the amount of extra work that AMP might introduce within your team from a learning and maintenance perspective;
• you should consider your audience. Doint optimizations within your regular platform or architacture will give you more flexibility to even optimize further;
• scaling: the homepage of amp.dev is using 14 JavaScript files. This lead to performance bottleneck and sub-par First Input Delay data. If you need to use more libraries for your specific needs, you might yet again run into performance issues.