You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
One consequence of our use of JSON Schema (and therefore JSON) to represent XDM is that the concept of an "object" (a structure with named fields that have a value) and a "map" (a container that maintains a relationship between keys and values) are not distinct.
While in JSON objects and maps can be treated (nearly) interchangeably, in XDM this is not true because we interpret XDM descriptions as describing a schema. In non-JSON environments (for example, a Parquet file), the properties of objects are mapped to columns, and the values of those properties are mapped to rows or entries.
While this is usually how we model data, there are many cases where we actually want the semantics of a map, where the keys are not columns but are themselves data elements that are stored in the rows/entries/etc. Environments such as a Parquet file support a logical map type, but we don't have any way to express in XDM that an object should be treated as a map.
Note this issue is dependent on #493. We should extend the current set of XDM types (which just need to be documented) with the map type.
What are the schemas that are affected by the issue
All
What are examples of products that are impacted by the issue
Adobe Audience Manager
The text was updated successfully, but these errors were encountered:
@lrosenthol unfortunately many of the data processing technologies we rely heavily on (such as Spark, and Parquet) simply don't work that way. Object properties get mapped to columns/schema, while maps are a distinct construct represented in the data. Maps are far less efficient and performant in these environments, so for most usages we want the column-based representation, but we do need maps for some use cases. We need something in XDM that allows us to signal when a map is desired rather than columns.
One consequence of our use of JSON Schema (and therefore JSON) to represent XDM is that the concept of an "object" (a structure with named fields that have a value) and a "map" (a container that maintains a relationship between keys and values) are not distinct.
While in JSON objects and maps can be treated (nearly) interchangeably, in XDM this is not true because we interpret XDM descriptions as describing a schema. In non-JSON environments (for example, a Parquet file), the properties of objects are mapped to columns, and the values of those properties are mapped to rows or entries.
While this is usually how we model data, there are many cases where we actually want the semantics of a map, where the keys are not columns but are themselves data elements that are stored in the rows/entries/etc. Environments such as a Parquet file support a logical map type, but we don't have any way to express in XDM that an object should be treated as a map.
Note this issue is dependent on #493. We should extend the current set of XDM types (which just need to be documented) with the map type.
What are the schemas that are affected by the issue
All
What are examples of products that are impacted by the issue
Adobe Audience Manager
The text was updated successfully, but these errors were encountered: