Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Avoid emitting YAML back-references in kpt fn output #183

Merged
merged 2 commits into from
Aug 20, 2020

Conversation

dflemstr
Copy link
Contributor

@dflemstr dflemstr commented Aug 13, 2020

Avoid using YAML refs because many YAML parsers in the k8s ecosystem don't support them.

Without this change, a typical output from a kpt fn run might look like this:

apiVersion: v1
kind: ResourceList
metadata:
  name: output
items:
- apiVersion: v1
  kind: Foo
  metadata:
    name: bar
  spec:
    array: &ref_0
    - &ref_1
      baz: 1
results:
- message: something is wrong
  severity: error
  resourceRef:
    apiVersion: v1
    kind: Foo
    namespace: ''
    name: bar
  file: {}
  field:
    path: spec.array
    currentValue: *ref_0
    suggestedValue:
    - *ref_1
    - baz: 3

The usage of &ref_0 and *ref_0 breaks many YAML parsers that don't implement that YAML feature.

@prachirp
Copy link
Contributor

prachirp commented Aug 13, 2020

@dflemstr I didn't know about refs before so thanks for bringing attention to them. I think back references fit very well with Don't Repeat Yourself principle. This seems like something that will come default in yaml parsers in 2-3 yrs, esp since js-yaml already defaults noRefs to false. WDYT? Is this accurate?

If so, we should track an issue to revisit this before v1.0.0 release.

@dflemstr
Copy link
Contributor Author

dflemstr commented Aug 14, 2020

@prachirp I agree that in principle anchors/back-references have some benefits in that they do aid in DRY. In practice, this seems to be one of the obscure features in the YAML spec that even many popular YAML parsers have a hard time supporting correctly.

For example, one of our tools is using the Jackson library and is hitting this bug: FasterXML/jackson-dataformats-text#98 which seems to have affected other popular products in the ecosystem like Spinnaker/Keel and Elasticsearch. Despite many attempts to fix it, it seems that even the most popular Java YAML parsing libraries, Jackson and SnakeYAML, both still have trouble supporting this feature.

I'm not sure if I'd expect this situation to change any time soon. And, especially since kpt function ResourceLists are supposed to be machine readable, I would vote for applying the Robustness Principle in this case!

Copy link
Contributor

@prachirp prachirp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dflemstr I see, thanks for the background! I agree, robustness comes first.

LGTM, but let's wait for @frankfarzan to take a look too.

@dflemstr
Copy link
Contributor Author

@frankfarzan would you have a chance to take a look at this? :)

Copy link
Contributor

@frankfarzan frankfarzan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Look good, just a minor comment.

@@ -403,13 +403,13 @@ export interface Result {
*/
export type Severity = 'error' | 'warn' | 'info';

interface JsonArray extends Array<Json> {}
export interface JsonArray extends Array<Json> {}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All exported types need to be documented with a comment.

@dflemstr
Copy link
Contributor Author

@frankfarzan done!

@frankfarzan frankfarzan merged commit e5e3654 into GoogleContainerTools:master Aug 20, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants