Feature Request: No way to check AMP Validation State using Vanilla HTML+JS #9371

johnnyshankman · 2017-05-16T17:49:32Z

What's the issue?

Feature request:

Gherkin:
As a developer writing end to end tests
When I go to an AMP page with #development=1 appended to the URL
I expect that I can check the validation state of the current page using a single line of synchronous JS in the console

More info
I think it makes the most sense for there to be a data attribute appended to the <html amp> element in this fashion <html amp amp-valid> and for failure <html amp amp-invalid>, but only when in development mode.

That way the developer can reliably check that attribute in order to assert the validation state of the AMP page without any GET requests to third parties, XHR, or Node.

I envision using something like
return document.documentElement.hasAttribute('amp-valid') in my Ghost Inspector tests or in my Selenium/Capybara feature tests. I'd just have to account for the 1-2 seconds it takes for the validator to run before asserting.

How do we reproduce the issue?

N/A

What browsers are affected?

N/A

Which AMP version is affected?

Latest 1494446565961

The text was updated successfully, but these errors were encountered:

jridgewell · 2017-05-16T19:01:35Z

Why aren't you just using the validator?

johnnyshankman · 2017-05-17T19:16:53Z

@jridgewell not possible from something like Ghost Inspector, or Ruby/Selinum based feature testing. I have an already-renderd AMP page that I've asserted has correct content and want to lastly check that it is amp-valid.

We're not testing against pre-rendered fixtures, we're using this for production smoke testing real articles in using Selenium to keep tabs on our CI.

Use case: Every hour Ghost Inspector hits production AMP article and appends #development=1, checks the response for 200, checks for some content, then finally checks that the document is amp-valid using this new feature.

jridgewell · 2017-05-17T22:05:55Z

Can you use Cloudflare's validation API?

johnnyshankman · 2017-05-18T02:57:40Z

@jridgewell Yes, I could write a separate test suite that solely hits Cloudflare's validation API endpoint and then parses the JSON to check for validity (although not in Ghost Inspector, it'd have to be done using a different tool).

This seems like a redundant/unnecessary step to me though, since I'm already running E2E feature tests against fully-rendered AMP pages in Selenium/Capybara (or alternatively Ghost Inspector).

Wouldn't it be more efficient to take my same E2E tests with fully rendered AMP pages but run them against pages in #development=1 mode and then run Capybara's evaluate_script to assert something like window.AMP.pageIsValid becoming true when the time comes?

That way I get clean E2E feature tests which test for correct content and amp-validity without having to do any asynchronous API calls to 3rd party services.

Here's the Capybara method I'm hoping to leverage in this scenario and here's a similar Ghost Inspector feature that would also do the job.

Definitely open to discussion though! Let me know of your concerns etc, I'm all ears.

johnnyshankman · 2017-05-18T18:41:38Z

Another reason why using Cloudflare isn't a perfect solution:

We host our CI/pre-prod builds on a private server requiring username+password credentials. We'd have to expose our credentials into the URL using https://amp.cloudflare.com/q/name:pwd@url.com syntax in order to let Cloudflare through our authentication and validate the page. Not ideal.

@jridgewell

jridgewell · 2017-05-18T18:56:05Z

/to @powdercloud. Ideally, the validator included with #development=1 would have just have a validate that works like the NPM module.

powdercloud · 2017-05-19T00:06:45Z

I looked into it for a bit. The status quo is that when #development=1 is appended, this code runs:
https://github.com/ampproject/amphtml/blob/master/src/validator-integration.js
Specifically, there's a call to amp.validator.validateUrlAndLog.
And it looks like once this is all done and I'm in the browser's console (I use Chrome), I can put
amp.validator.validateUrlAndLog(window.location.href);
to trigger it again. The method validateUrlAndLog itself is in here:
https://github.com/ampproject/amphtml/blob/master/validator/engine/validator-in-browser.js
The main thing that may surprise at first is that it does a HTTP get request to fetch the page a second time. This is necessary because the DOM from which it was loaded is heavily modified by the AMP runtime once the validator is there, so we really need to get the contents of the page again.

If amp.validator.validateUrlAndLog were to also store a result object, would this be sufficient?
Then you could access this with amp.validator.validationResult. It would have all the information, like the status, the errors, parameters for the errors, etc.

The way to graft this together is for amp.validator.validateUrlAndLog to return this object, and for the runtime to store it in this variable. It's a slight bit hacky but makes it so we don't have to add this global variable to engine.js.

Opinions on this most welcome. I'm not familiar with Ghost Inspector etc., so it's hard for me to say whether this is the simplest possible solution - but maybe better than annotating the DOM. Note that with the solution outlined here you'd still end up with an extra HTTP fetch per document you're checking - that's just how #development=1 works.

johnnyshankman · 2017-06-01T14:51:32Z

@powdercloud I noticed the same method and found the same confusion, why is there no function to revalidate the current page and return the state synchronously?

validateUrlAndLog is okay, but not nearly as useful because it is async/promise-based (due to the underlying GET request). Things like ghost inspector don't work well with promises are async.

Storing the return value in a global variable would be better than the current implementation because our front end tests can always sleep and then check that variable. I definitely like the simplicity of that idea, and it's just as easy as my original idea of amp-valid as a class on the HTML.

What would be most ideal is validateCurrentDocumentAndLog. It would validate the current page and synchronously return true or false, but I know that's much more complex.

powdercloud · 2017-06-05T20:04:04Z

This might be a way.

Load the AMP page. E.g. load this one, with development=1 so that the validator gets loaded:
https://www.ampproject.org/docs/tutorials/login_requiring#development=1
In the console, run this:
var request = new XMLHttpRequest(); request.open('GET', location.href, false); request.send(null); var validationResult = amp.validator.validateString(request.responseText);
validationResult has the status field (in this case 'PASS') and also the errors if there are any.

Also:

On my (pretty big) screen this fits into one line ...
Chrome prints a deprecation warning for making a synchronous request (the last parameter, false, to request.open). Also documented here: https://developer.mozilla.org/en-US/docs/Web/API/XMLHttpRequest/Synchronous_and_Asynchronous_Requests
It would be worthwhile checking the other fields of request as well, e.g. request.status and make sure it's 200 before proceeding.
This scheme fetches the page three times. But perhaps for testing this is a non-issue. If it bothers you, you could make a server side handler on your end which script includes the validator.js and then has a <script> block which runs the code above to invoke it. The server side handler could take the URL / page as a parameter e.g. it could be /amp-validate?doc=path/to/amp.html or similar and if you're making server side stuff anyway this should be easy to do.

johnnyshankman · 2017-06-06T20:15:20Z

@powdercloud that works great with one minor addition.

var request = new XMLHttpRequest(); 
request.open('GET', location.href, false); 
request.send(null); 
var validationResult = amp.validator.validateString(request.responseText);
var valid = validationResult.status === "PASS";

You are right that it is inefficient but this gets the job done for now!

Assuming the underlying implementation of the validator doesn't change suddenly we'll be a-ok.

If it does, which it could since this is undocumented/unsupported functionality, my tests will start failing and I'll have to refactor or remove the step.

It's unfortunate that I have to re-request the page I'm already on but at the same time, you are right this is indeed a testing environment where making one extra request isn't game-breaking.

powdercloud · 2017-06-07T00:08:22Z

amp.validator.validateString(inputText) isn't going to change, and neither is the status field of the return value; if we make changes they'll be backwards compatible by necessity because the runtime, the npm library, the chrome extension, etc. depends on this. I also don't see why the behavior of loading this for #development=1 would change.
@johnnyshankman It looks like we're both happy so I'll take the liberty and close this bug. Please feel free to reopen or make another bug as you see fit. Thank you.

praharshjain · 2018-10-12T20:19:04Z

As of October 13, 2018, AMP validation can be checked using -
window.AMPErrors.length === 0
(supported in https://cdn.ampproject.org/v0.js)

johnnyshankman · 2019-01-18T16:38:06Z

Not true, just checked a page with errors and window.AMPErrors was []. I think that's used internally for something else @praharshjain

Ladi-Harish · 2019-08-27T04:28:36Z

I think you will get some solution to using this website AMP Validator.

By using this website you can test multiple numbers of URLs in a short time. And this site is secure, accurate, fast and easy to use. The most interesting thing is you will get the proper beautified email report.

johnnyshankman changed the title ~~No way to check AMP Validation State using Vanilla HTML+JS~~ Feature Request: No way to check AMP Validation State using Vanilla HTML+JS May 16, 2017

jridgewell added the Type: Feature Request label May 16, 2017

jridgewell added this to the Pending Triage milestone May 16, 2017

jridgewell added the Category: Validator label May 18, 2017

jridgewell assigned powdercloud May 18, 2017

powdercloud closed this as completed Jun 7, 2017

ampwafflebot removed the Type: Feature Request label Jun 7, 2017

kienstra mentioned this issue Jan 25, 2018

[WIP] Issue #843: Validate Gutenberg blocks for AMP compliance ampproject/amp-wp#902

Closed

honeybadgerdontcare added WG: caching and removed Category: Validator labels Nov 11, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Feature Request: No way to check AMP Validation State using Vanilla HTML+JS #9371

Feature Request: No way to check AMP Validation State using Vanilla HTML+JS #9371

johnnyshankman commented May 16, 2017 •

edited

Loading

jridgewell commented May 16, 2017

johnnyshankman commented May 17, 2017 •

edited

Loading

jridgewell commented May 17, 2017

johnnyshankman commented May 18, 2017 •

edited

Loading

johnnyshankman commented May 18, 2017 •

edited

Loading

jridgewell commented May 18, 2017

powdercloud commented May 19, 2017 •

edited

Loading

johnnyshankman commented Jun 1, 2017 •

edited

Loading

powdercloud commented Jun 5, 2017 •

edited

Loading

johnnyshankman commented Jun 6, 2017 •

edited

Loading

powdercloud commented Jun 7, 2017

praharshjain commented Oct 12, 2018 •

edited

Loading

johnnyshankman commented Jan 18, 2019

Ladi-Harish commented Aug 27, 2019

Feature Request: No way to check AMP Validation State using Vanilla HTML+JS #9371

Feature Request: No way to check AMP Validation State using Vanilla HTML+JS #9371

Comments

johnnyshankman commented May 16, 2017 • edited Loading

What's the issue?

How do we reproduce the issue?

What browsers are affected?

Which AMP version is affected?

jridgewell commented May 16, 2017

johnnyshankman commented May 17, 2017 • edited Loading

jridgewell commented May 17, 2017

johnnyshankman commented May 18, 2017 • edited Loading

johnnyshankman commented May 18, 2017 • edited Loading

jridgewell commented May 18, 2017

powdercloud commented May 19, 2017 • edited Loading

johnnyshankman commented Jun 1, 2017 • edited Loading

powdercloud commented Jun 5, 2017 • edited Loading

johnnyshankman commented Jun 6, 2017 • edited Loading

powdercloud commented Jun 7, 2017

praharshjain commented Oct 12, 2018 • edited Loading

johnnyshankman commented Jan 18, 2019

Ladi-Harish commented Aug 27, 2019

johnnyshankman commented May 16, 2017 •

edited

Loading

johnnyshankman commented May 17, 2017 •

edited

Loading

johnnyshankman commented May 18, 2017 •

edited

Loading

johnnyshankman commented May 18, 2017 •

edited

Loading

powdercloud commented May 19, 2017 •

edited

Loading

johnnyshankman commented Jun 1, 2017 •

edited

Loading

powdercloud commented Jun 5, 2017 •

edited

Loading

johnnyshankman commented Jun 6, 2017 •

edited

Loading

praharshjain commented Oct 12, 2018 •

edited

Loading