Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GTD-2: API reference improvements #48

Merged

Conversation

lewisnyman
Copy link
Contributor

The user-facing changes in this PR:

  • enum parameter types render correctly
  • When printing a single API path, all referenced schemas are now rendered below that path
  • Schemas parameters that reference other schemas now render links to those schemas
  • Example response bodies are now rendered in JSON
  • allOf is now supported in schemas, these will be rendered correctly

Linting and updated tests will come later.

lewisnyman and others added 25 commits August 23, 2018 12:14
@lewisnyman lewisnyman requested a review from heathd August 30, 2018 16:32
@jonathanglassman
Copy link
Contributor

@MatMoore tagging you to review as well

@MatMoore
Copy link
Contributor

I haven't done a full review yet, but I've tested this branch and found a couple of bugs related to the examples. I'll list them here but let me know if it's easier to report these offline.

A response without a schema cannot be rendered

Tested with https://github.com/OAI/OpenAPI-Specification/blob/master/examples/v3.0/api-with-examples.yaml

As I understand it, this is not the specific kind of example you implemented to make the pay spec work, but I would expect this spec to render, even if it doesn't show the examples. In this spec the responses contain examples, but don't contain schemas. When I tried to render this it threw this exception:

NoMethodError: undefined method `properties' for nil:NilClass
	/Users/matmoore/govuk/tech-docs-gem/lib/govuk_tech_docs/api_reference/api_reference_renderer.rb:173:in `schema_properties'
	/Users/matmoore/govuk/tech-docs-gem/lib/govuk_tech_docs/api_reference/api_reference_renderer.rb:168:in `json_output'
	(erb):14:in `block in responses'
	/Users/matmoore/.rbenv/versions/2.4.2/lib/ruby/2.4.0/forwardable.rb:229:in `each'
	/Users/matmoore/.rbenv/versions/2.4.2/lib/ruby/2.4.0/forwardable.rb:229:in `each'
	(erb):8:in `responses'
	/Users/matmoore/.rbenv/versions/2.4.2/lib/ruby/2.4.0/erb.rb:896:in `eval'
	/Users/matmoore/.rbenv/versions/2.4.2/lib/ruby/2.4.0/erb.rb:896:in `result'
	/Users/matmoore/govuk/tech-docs-gem/lib/govuk_tech_docs/api_reference/api_reference_renderer.rb:157:in `responses'
	/Users/matmoore/govuk/tech-docs-gem/lib/govuk_tech_docs/api_reference/api_reference_renderer.rb:141:in `block in operations'
	/Users/matmoore/govuk/tech-docs-gem/lib/govuk_tech_docs/api_reference/api_reference_renderer.rb:138:in `each'
	/Users/matmoore/govuk/tech-docs-gem/lib/govuk_tech_docs/api_reference/api_reference_renderer.rb:138:in `operations'
	/Users/matmoore/govuk/tech-docs-gem/lib/govuk_tech_docs/api_reference/api_reference_renderer.rb:41:in `path'
	/Users/matmoore/govuk/tech-docs-gem/lib/govuk_tech_docs/api_reference/api_reference_renderer.rb:27:in `block in api_full'
	/Users/matmoore/.rbenv/versions/2.4.2/lib/ruby/2.4.0/forwardable.rb:229:in `each'
	/Users/matmoore/.rbenv/versions/2.4.2/lib/ruby/2.4.0/forwardable.rb:229:in `each'
	/Users/matmoore/govuk/tech-docs-gem/lib/govuk_tech_docs/api_reference/api_reference_renderer.rb:23:in `api_full'
	/Users/matmoore/govuk/tech-docs-gem/lib/govuk_tech_docs/api_reference/api_reference_extension.rb:69:in `api'
	/Users/matmoore/govuk/tech-docs-gem/lib/govuk_tech_docs/tech_docs_html_renderer.rb:14:in `paragraph'

Inconsistent markdown rendering

The spec says "Throughout the specification description fields are noted as supporting CommonMark markdown formatting. Where OpenAPI tooling renders rich text it MUST support, at a minimum, markdown syntax as described by CommonMark 0.27. Tooling MAY choose to ignore some CommonMark features to address security concerns."

However I noticed that the content api spec (after making the changes suggested in alphagov/content-store#460) does not render some markdown in descriptions.

screen shot 2018-08-31 at 16 29 01

We do render some markdown though, I noticed that in the pay spec YOUR_API_KEY renders in italics, so we should probably update that spec to avoid that.

Incorrect examples and/or schema names

This may be a combination of multiple things. I've noticed in several schemas that examples don't render properly and/or the documentation shows a JSON path as the schema name:

It might be something to do with one of these things:

  • The schema being defined inline instead of being a reference to an object in compenents
  • Schemas using items or additionalProperties instead of just properties

screen shot 2018-08-31 at 16 04 14

screen shot 2018-08-31 at 16 43 44

screen shot 2018-08-31 at 16 48 11

Schemas not rendering

In the content performance manager spec (see above) I also got entire schemas not rendering. I suspect this is due to using additionalProperties to define an object that maps arbitrary keys to values of a single type.

For example:

    TimeSeriesResponse:
        description: "An object where each key is a metric ID and each value is a time series."
        type: object
        additionalProperties:
          $ref: '#/components/schemas/TimeSeries'

This currently renders using widdershins (https://content-performance-api.publishing.service.gov.uk/reference.html#timeseriesresponse) so we should be able to get similar results from this implementation.

Copy link
Contributor

@MatMoore MatMoore left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've had a look through the commits and just have a couple of minor comments about the code.

My main concern is that the rendering of schema objects and example responses doesn't handle some of the other specs we need to be able to render (see above).

That said, I'd prefer to merge this PR as is and then tackle those bugs than do everything as part of one PR, as this work fixes a bunch of issues on master.

@@ -106,6 +106,14 @@ def schemas_from_schema(schema)
if schema.type == 'array'
properties.push schema.items
end
allOf = schema["allOf"]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor thing but allOf is camel case here whereas the convention is to use snake case.

Also, do you think it would make sense to handle anyOf/oneOf in the same way as this? Ideally the design of the template should make the semantics clear, but I'm wondering if just showing all the properties is better than doing nothing for these.

properties = schema.properties
properties.each do |key, property|
properties = []
schema.properties.each do |property|
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is schema.properties a hash? If so you can just do properties = schema.properties.values.

@heathd
Copy link
Contributor

heathd commented Sep 4, 2018

Just starting to look over.

I'm sure you noticed that the build failed due to some rubocop failures.

Running the specs locally I also got one spec failure:

Failures:

  1) The tech docs template generates a working static site
     Failure/Error: expect(page).to have_css('h2#get-pets', text: 'GET /pets')
       expected #has_css?("h2#get-pets", {:text=>"GET /pets", :session_options=>#<Capybara::ReadOnlySessionConfig(#<Capybara::SessionConfig:0x...y=false, @wait_on_first_by_default=false, @automatic_label_click=false, @enable_aria_label=false>)>}) to return true, got false
     # ./spec/features/integration_spec.rb:139:in `then_there_is_correct_api_path_content'
     # ./spec/features/integration_spec.rb:36:in `block (2 levels) in <top (required)>'

Finished in 20.42 seconds (files took 1.09 seconds to load)
42 examples, 1 failure

Failed examples:

rspec ./spec/features/integration_spec.rb:9 # The tech docs template generates a working static site

haven't dug into it

@heathd
Copy link
Contributor

heathd commented Sep 4, 2018

I've been trying to get the pay oais3 spec to parse and getting these errors:

$ bundle exec middleman build
   identical  build/stylesheets/print.css
   identical  build/stylesheets/screen-old-ie.css
   identical  build/stylesheets/screen.css
   identical  build/images/govuk-crest.png
   identical  build/images/anchored-heading-icon.png
   identical  build/images/anchored-heading-icon-2x.png
   identical  build/images/govuk-icn-close.png
   identical  build/images/gov.uk_logotype_crown_invert_trans.png
   identical  build/images/govuk-icn-numbered-list.png
   identical  build/images/open-government-licence.png
   identical  build/images/govuk-icn-numbered-list@2x.png
   identical  build/images/open-government-licence_2x.png
   identical  build/images/gov.uk_logotype_crown.png
   identical  build/favicon.ico
   identical  build/images/govuk-icn-close@2x.png
   identical  build/images/govuk-crest-2x.png
   identical  build/images/gov.uk_logotype_crown-2x.png
   identical  build/images/search-result-caret.svg
   identical  build/api/pages.json
       error  build/api-reference.html
Error processing resource for index: api-reference.html
Invalid data for #/paths/%2Fv1%2Fpayments/get/responses/200/%24ref. Reference does not resolve to a valid object
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/openapi3_parser-0.5.2/lib/openapi3_parser/node_factory/field.rb:110:in `validate'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/openapi3_parser-0.5.2/lib/openapi3_parser/node_factory/field.rb:86:in `data'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/openapi3_parser-0.5.2/lib/openapi3_parser/node_factory/field.rb:65:in `data'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/openapi3_parser-0.5.2/lib/openapi3_parser/node_factory/field.rb:44:in `node'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/openapi3_parser-0.5.2/lib/openapi3_parser/node_factory/reference.rb:33:in `build_node'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/openapi3_parser-0.5.2/lib/openapi3_parser/node_factory/object.rb:47:in `node'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/openapi3_parser-0.5.2/lib/openapi3_parser/node_factory/map.rb:134:in `block in data'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/openapi3_parser-0.5.2/lib/openapi3_parser/node_factory/map.rb:133:in `each'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/openapi3_parser-0.5.2/lib/openapi3_parser/node_factory/map.rb:133:in `each_with_object'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/openapi3_parser-0.5.2/lib/openapi3_parser/node_factory/map.rb:133:in `data'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/openapi3_parser-0.5.2/lib/openapi3_parser/node_factory/map.rb:109:in `data'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/openapi3_parser-0.5.2/lib/openapi3_parser/node_factory/map.rb:51:in `node'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/openapi3_parser-0.5.2/lib/openapi3_parser/node_factory/object_factory/node_builder.rb:67:in `resolve_value'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/openapi3_parser-0.5.2/lib/openapi3_parser/node_factory/object_factory/node_builder.rb:60:in `block in build_node_data'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/openapi3_parser-0.5.2/lib/openapi3_parser/node_factory/object_factory/node_builder.rb:56:in `each'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/openapi3_parser-0.5.2/lib/openapi3_parser/node_factory/object_factory/node_builder.rb:56:in `each_with_object'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/openapi3_parser-0.5.2/lib/openapi3_parser/node_factory/object_factory/node_builder.rb:56:in `build_node_data'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/openapi3_parser-0.5.2/lib/openapi3_parser/node_factory/object_factory/node_builder.rb:36:in `node_data'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/openapi3_parser-0.5.2/lib/openapi3_parser/node_factory/object_factory/node_builder.rb:12:in `node_data'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/openapi3_parser-0.5.2/lib/openapi3_parser/node_factory/object.rb:103:in `build_node'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/openapi3_parser-0.5.2/lib/openapi3_parser/node_factory/object.rb:47:in `node'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/openapi3_parser-0.5.2/lib/openapi3_parser/node_factory/object_factory/node_builder.rb:67:in `resolve_value'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/openapi3_parser-0.5.2/lib/openapi3_parser/node_factory/object_factory/node_builder.rb:60:in `block in build_node_data'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/openapi3_parser-0.5.2/lib/openapi3_parser/node_factory/object_factory/node_builder.rb:56:in `each'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/openapi3_parser-0.5.2/lib/openapi3_parser/node_factory/object_factory/node_builder.rb:56:in `each_with_object'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/openapi3_parser-0.5.2/lib/openapi3_parser/node_factory/object_factory/node_builder.rb:56:in `build_node_data'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/openapi3_parser-0.5.2/lib/openapi3_parser/node_factory/object_factory/node_builder.rb:36:in `node_data'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/openapi3_parser-0.5.2/lib/openapi3_parser/node_factory/object_factory/node_builder.rb:12:in `node_data'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/openapi3_parser-0.5.2/lib/openapi3_parser/node_factory/object.rb:103:in `build_node'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/openapi3_parser-0.5.2/lib/openapi3_parser/node_factory/object.rb:47:in `node'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/openapi3_parser-0.5.2/lib/openapi3_parser/node_factory/map.rb:134:in `block in data'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/openapi3_parser-0.5.2/lib/openapi3_parser/node_factory/map.rb:133:in `each'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/openapi3_parser-0.5.2/lib/openapi3_parser/node_factory/map.rb:133:in `each_with_object'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/openapi3_parser-0.5.2/lib/openapi3_parser/node_factory/map.rb:133:in `data'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/openapi3_parser-0.5.2/lib/openapi3_parser/node_factory/map.rb:109:in `data'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/openapi3_parser-0.5.2/lib/openapi3_parser/node_factory/map.rb:51:in `node'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/openapi3_parser-0.5.2/lib/openapi3_parser/node_factory/object_factory/node_builder.rb:67:in `resolve_value'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/openapi3_parser-0.5.2/lib/openapi3_parser/node_factory/object_factory/node_builder.rb:60:in `block in build_node_data'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/openapi3_parser-0.5.2/lib/openapi3_parser/node_factory/object_factory/node_builder.rb:56:in `each'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/openapi3_parser-0.5.2/lib/openapi3_parser/node_factory/object_factory/node_builder.rb:56:in `each_with_object'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/openapi3_parser-0.5.2/lib/openapi3_parser/node_factory/object_factory/node_builder.rb:56:in `build_node_data'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/openapi3_parser-0.5.2/lib/openapi3_parser/node_factory/object_factory/node_builder.rb:36:in `node_data'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/openapi3_parser-0.5.2/lib/openapi3_parser/node_factory/object_factory/node_builder.rb:12:in `node_data'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/openapi3_parser-0.5.2/lib/openapi3_parser/node_factory/object.rb:103:in `build_node'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/openapi3_parser-0.5.2/lib/openapi3_parser/node_factory/object.rb:47:in `node'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/openapi3_parser-0.5.2/lib/openapi3_parser/document.rb:93:in `root'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/2.4.0/forwardable.rb:223:in `info'
 /Users/davidheath/gds/convivio-tech-docs-gem/lib/govuk_tech_docs/api_reference/api_reference_extension.rb:89:in `api_info'
 /Users/davidheath/gds/convivio-tech-docs-gem/lib/govuk_tech_docs/api_reference/api_reference_extension.rb:69:in `api'
 /Users/davidheath/gds/convivio-tech-docs-gem/lib/govuk_tech_docs/tech_docs_html_renderer.rb:14:in `paragraph'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/middleman-core-4.2.1/lib/middleman-core/renderers/redcarpet.rb:51:in `render'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/middleman-core-4.2.1/lib/middleman-core/renderers/redcarpet.rb:51:in `evaluate'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/tilt-2.0.8/lib/tilt/template.rb:109:in `render'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/middleman-core-4.2.1/lib/middleman-core/file_renderer.rb:79:in `render'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/contracts-0.13.0/lib/contracts/method_reference.rb:43:in `send_to'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/contracts-0.13.0/lib/contracts/call_with.rb:76:in `call_with'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/contracts-0.13.0/lib/contracts/method_handler.rb:138:in `block in redefine_method'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/middleman-core-4.2.1/lib/middleman-core/template_renderer.rb:184:in `_render_with_all_renderers'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/middleman-core-4.2.1/lib/middleman-core/template_renderer.rb:147:in `block in render'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/activesupport-5.0.7/lib/active_support/notifications.rb:166:in `instrument'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/middleman-core-4.2.1/lib/middleman-core/util.rb:21:in `instrument'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/middleman-core-4.2.1/lib/middleman-core/template_renderer.rb:146:in `render'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/contracts-0.13.0/lib/contracts/method_reference.rb:43:in `send_to'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/contracts-0.13.0/lib/contracts/call_with.rb:76:in `call_with'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/contracts-0.13.0/lib/contracts/method_handler.rb:138:in `block in redefine_method'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/middleman-core-4.2.1/lib/middleman-core/sitemap/resource.rb:154:in `render'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/contracts-0.13.0/lib/contracts/method_reference.rb:43:in `send_to'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/contracts-0.13.0/lib/contracts/call_with.rb:76:in `call_with'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/contracts-0.13.0/lib/contracts/method_handler.rb:138:in `block in redefine_method'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/bundler/gems/middleman-search-50d072378c6f/lib/middleman-search/search-index-resource.rb:132:in `value_for'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/bundler/gems/middleman-search-50d072378c6f/lib/middleman-search/search-index-resource.rb:97:in `block (3 levels) in build_index'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/bundler/gems/middleman-search-50d072378c6f/lib/middleman-search/search-index-resource.rb:96:in `each'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/bundler/gems/middleman-search-50d072378c6f/lib/middleman-search/search-index-resource.rb:96:in `block (2 levels) in build_index'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/bundler/gems/middleman-search-50d072378c6f/lib/middleman-search/search-index-resource.rb:89:in `catch'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/bundler/gems/middleman-search-50d072378c6f/lib/middleman-search/search-index-resource.rb:89:in `block in build_index'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/bundler/gems/middleman-search-50d072378c6f/lib/middleman-search/search-index-resource.rb:87:in `each'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/bundler/gems/middleman-search-50d072378c6f/lib/middleman-search/search-index-resource.rb:87:in `each_with_index'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/bundler/gems/middleman-search-50d072378c6f/lib/middleman-search/search-index-resource.rb:87:in `build_index'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/bundler/gems/middleman-search-50d072378c6f/lib/middleman-search/search-index-resource.rb:31:in `render'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/middleman-core-4.2.1/lib/middleman-core/rack.rb:112:in `process_request'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/middleman-core-4.2.1/lib/middleman-core/rack.rb:66:in `block in call'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/middleman-core-4.2.1/lib/middleman-core/rack.rb:65:in `catch'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/middleman-core-4.2.1/lib/middleman-core/rack.rb:65:in `call'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/rack-2.0.5/lib/rack/urlmap.rb:68:in `block in call'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/rack-2.0.5/lib/rack/urlmap.rb:53:in `each'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/rack-2.0.5/lib/rack/urlmap.rb:53:in `call'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/middleman-core-4.2.1/lib/middleman-core/extensions/minify_javascript.rb:55:in `call'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/middleman-core-4.2.1/lib/middleman-core/extensions/minify_css.rb:63:in `call'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/middleman-autoprefixer-2.7.1/lib/middleman-autoprefixer/extension.rb:48:in `call'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/rack-2.0.5/lib/rack/head.rb:12:in `call'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/rack-2.0.5/lib/rack/lint.rb:49:in `_call'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/rack-2.0.5/lib/rack/lint.rb:37:in `call'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/rack-2.0.5/lib/rack/builder.rb:153:in `call'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/rack-2.0.5/lib/rack/mock.rb:74:in `request'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/rack-2.0.5/lib/rack/mock.rb:56:in `get'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/middleman-core-4.2.1/lib/middleman-core/builder.rb:232:in `block in output_resource'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/activesupport-5.0.7/lib/active_support/notifications.rb:166:in `instrument'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/middleman-core-4.2.1/lib/middleman-core/util.rb:21:in `instrument'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/middleman-core-4.2.1/lib/middleman-core/builder.rb:225:in `output_resource'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/contracts-0.13.0/lib/contracts/method_reference.rb:43:in `send_to'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/contracts-0.13.0/lib/contracts/call_with.rb:76:in `call_with'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/contracts-0.13.0/lib/contracts/method_handler.rb:138:in `block in redefine_method'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/parallel-1.12.1/lib/parallel.rb:484:in `call_with_index'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/parallel-1.12.1/lib/parallel.rb:455:in `process_incoming_jobs'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/parallel-1.12.1/lib/parallel.rb:437:in `block in worker'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/parallel-1.12.1/lib/parallel.rb:428:in `fork'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/parallel-1.12.1/lib/parallel.rb:428:in `worker'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/parallel-1.12.1/lib/parallel.rb:419:in `block in create_workers'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/parallel-1.12.1/lib/parallel.rb:418:in `each'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/parallel-1.12.1/lib/parallel.rb:418:in `each_with_index'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/parallel-1.12.1/lib/parallel.rb:418:in `create_workers'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/parallel-1.12.1/lib/parallel.rb:358:in `work_in_processes'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/parallel-1.12.1/lib/parallel.rb:264:in `map'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/middleman-core-4.2.1/lib/middleman-core/builder.rb:137:in `output_resources'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/contracts-0.13.0/lib/contracts/method_reference.rb:43:in `send_to'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/contracts-0.13.0/lib/contracts/call_with.rb:76:in `call_with'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/contracts-0.13.0/lib/contracts/method_handler.rb:138:in `block in redefine_method'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/middleman-core-4.2.1/lib/middleman-core/builder.rb:131:in `output_files'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/contracts-0.13.0/lib/contracts/method_reference.rb:43:in `send_to'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/contracts-0.13.0/lib/contracts/call_with.rb:76:in `call_with'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/contracts-0.13.0/lib/contracts/method_handler.rb:138:in `block in redefine_method'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/middleman-core-4.2.1/lib/middleman-core/builder.rb:71:in `block in run!'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/activesupport-5.0.7/lib/active_support/notifications.rb:166:in `instrument'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/middleman-core-4.2.1/lib/middleman-core/util.rb:21:in `instrument'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/middleman-core-4.2.1/lib/middleman-core/builder.rb:70:in `run!'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/contracts-0.13.0/lib/contracts/method_reference.rb:43:in `send_to'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/contracts-0.13.0/lib/contracts/call_with.rb:76:in `call_with'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/contracts-0.13.0/lib/contracts/method_handler.rb:138:in `block in redefine_method'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/middleman-cli-4.2.1/lib/middleman-cli/build.rb:80:in `block in build'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/activesupport-5.0.7/lib/active_support/notifications.rb:166:in `instrument'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/middleman-core-4.2.1/lib/middleman-core/util.rb:21:in `instrument'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/middleman-cli-4.2.1/lib/middleman-cli/build.rb:79:in `build'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/thor-0.20.0/lib/thor/command.rb:27:in `run'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/thor-0.20.0/lib/thor/invocation.rb:126:in `invoke_command'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/thor-0.20.0/lib/thor/invocation.rb:133:in `block in invoke_all'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/thor-0.20.0/lib/thor/invocation.rb:133:in `each'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/thor-0.20.0/lib/thor/invocation.rb:133:in `map'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/thor-0.20.0/lib/thor/invocation.rb:133:in `invoke_all'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/thor-0.20.0/lib/thor/group.rb:232:in `dispatch'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/thor-0.20.0/lib/thor/invocation.rb:115:in `invoke'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/thor-0.20.0/lib/thor.rb:40:in `block in register'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/thor-0.20.0/lib/thor/command.rb:27:in `run'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/thor-0.20.0/lib/thor/invocation.rb:126:in `invoke_command'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/thor-0.20.0/lib/thor.rb:387:in `dispatch'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/thor-0.20.0/lib/thor/base.rb:466:in `start'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/middleman-cli-4.2.1/bin/middleman:70:in `<top (required)>'
 /Users/davidheath/.rbenv/versions/2.4.2/bin/middleman:23:in `load'
 /Users/davidheath/.rbenv/versions/2.4.2/bin/middleman:23:in `<top (required)>'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/bundler-1.16.3/lib/bundler/cli/exec.rb:74:in `load'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/bundler-1.16.3/lib/bundler/cli/exec.rb:74:in `kernel_load'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/bundler-1.16.3/lib/bundler/cli/exec.rb:28:in `run'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/bundler-1.16.3/lib/bundler/cli.rb:424:in `exec'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/bundler-1.16.3/lib/bundler/vendor/thor/lib/thor/command.rb:27:in `run'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/bundler-1.16.3/lib/bundler/vendor/thor/lib/thor/invocation.rb:126:in `invoke_command'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/bundler-1.16.3/lib/bundler/vendor/thor/lib/thor.rb:387:in `dispatch'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/bundler-1.16.3/lib/bundler/cli.rb:27:in `dispatch'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/bundler-1.16.3/lib/bundler/vendor/thor/lib/thor/base.rb:466:in `start'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/bundler-1.16.3/lib/bundler/cli.rb:18:in `start'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/bundler-1.16.3/exe/bundle:30:in `block in <top (required)>'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/bundler-1.16.3/lib/bundler/friendly_errors.rb:124:in `with_friendly_errors'
 /Users/davidheath/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/bundler-1.16.3/exe/bundle:22:in `<top (required)>'
 /Users/davidheath/.rbenv/versions/2.4.2/bin/bundle:23:in `load'
 /Users/davidheath/.rbenv/versions/2.4.2/bin/bundle:23:in `<main>'
       error  build/child-of-expired-page.html
       error  build/not-expired-page.html
       error  build/expired-page.html
   identical  build/core-layout.html
      create  build/pay-oais3.yml
   identical  build/petstore.yaml
   identical  build/search.json
       error  build/expired-page-with-owner.html
       error  build/headings.html
   identical  build/something/old.html
       error  build/index.html
   identical  build/something/old-as-well.html
       error  build/a-proxied-page.html
   identical  build/javascripts/application.js
There were errors during this build, re-run with `middleman build --verbose` to see the full exception.

@lewisnyman
Copy link
Contributor Author

@mikebell Is this the same validation error that you encountered before?

@mikebell
Copy link
Contributor

mikebell commented Sep 4, 2018

I'd suggest trying it with Ruby 2.5.1 and seeing if that works. I've not seen that specific error before I'm afraid.

@heathd
Copy link
Contributor

heathd commented Sep 4, 2018

Got the same error with 2.5.1

Error processing resource for index: api-reference.html
Invalid data for #/paths/%2Fv1%2Fpayments/get/responses/200/%24ref. Reference does not resolve to a valid object
 /Users/davidheath/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/openapi3_parser-0.5.2/lib/openapi3_parser/node_factory/field.rb:110:in `validate'
 /Users/davidheath/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/openapi3_parser-0.5.2/lib/openapi3_parser/node_factory/field.rb:86:in `data'

@mikebell
Copy link
Contributor

mikebell commented Sep 4, 2018

Try using this attached version. It fixes some validation issues with $ref that I found.

pay-oais3.zip

@heathd
Copy link
Contributor

heathd commented Sep 5, 2018

priorities:

  1. compound types using array eg:
    PaymentSearchResults:
      type: object
      properties:
        results:
          type: array
          readOnly: true
          items:
            $ref: '#/components/schemas/CardPayment'

we'd expect to see something like:

{
  "results": [
    {
      "amount": 1200,
      "state": {
        "status": "created",
        "finished": true,
        "message": "User cancelled the payment",
        "code": "P010"
      },
      "description": "Your Service Description",
      "reference": "your-reference",
      "email": "your email",
      "payment_id": "hu20sqlact5260q2nanm0q8u93",
      "payment_provider": "worldpay",
      "return_url": "http://your.service.domain/your-reference",
      "created_date": "2016-01-21T17:15:00Z",
      "refund_summary": {
        "status": "available",
        "amount_available": 0,
        "amount_submitted": 0
      },
      "settlement_summary": {
        "capture_submit_time": "2016-01-21T17:15:00Z",
        "captured_date": "2016-01-21"
      },
      "card_details": {
        "last_digits_card_number": "1234",
        "cardholder_name": "Mr. Card holder",
        "expiry_date": "12/20",
        "billing_address": {
          "line1": "address line 1",
          "line2": "address line 2",
          "postcode": "AB1 2CD",
          "city": "address city",
          "country": "UK"
        },
        "card_brand": "Visa"
      },
      "card_brand": "Visa"
    }
  ]
}

but actually get

results: {} 
  1. incorrect examples and/or schema names
  2. schemas not rendering

@@ -6,3 +6,6 @@ Naming/HeredocDelimiterNaming:

Lint/NestedMethodDefinition:
Enabled: false

Performance/HashEachMethods:
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Most objects constructed by openapi3_parser support .each but not .each_value but Rubocop treats it as a hash.

Copy link
Contributor

@MatMoore MatMoore Sep 7, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fair enough. Makes me wonder if we should add #each_value to the parser at some point though (cc @kevindew)

@lewisnyman
Copy link
Contributor Author

Tests should be passing now. It would be good to have more test coverage, including the request bodies, but the data structure of the pets.yml schema array are different to other files, and will need more work to get them rendering properly as well (see the code below). Ultimately it would be great to include pets-extended.yml with wider coverage of supported features.

pets.yml

Pets:
      type: array
      items:
        $ref: "#/components/schemas/Pet"

@MatMoore
Copy link
Contributor

Thanks @lewisnyman. I think the test coverage is something we can look at at a later date - I think it would definitely be helpful for the example generation given the complexity involved there.

@heathd if it's ok with you, I'd like to merge this and release a new version. It looks like some of the issues we raised earlier are now fixed, but I can raise github issues for anything that's left over and document any workarounds in the changelog.

@MatMoore MatMoore merged commit d3acc28 into alphagov:master Sep 12, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants