From 50abd444379f07483ea33bed7c751b4b572ca0de Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Jack=20=28=EC=A0=95=ED=99=98=29?= Date: Tue, 4 Jun 2019 13:09:48 -0700 Subject: [PATCH] Create AggregationDataStore module (#845) * Create AggregationDataStore module * Address Aaron's comments * Fix build failure AggregationDataStore: Schema (#846) * AggregationDataStore: Static Attribute Aggregation * Address comments * Implement TimeDimension and all its supporting components * refactor * Address comments from @aklish * Address comments from @aklish && Tests & Javadoc * Address comments from @aklish * Address comments from @aklish and @hellohanchen * Address comments from Aaron * ToMany is not supported * Address comments from Aaron Define QueryEngine Contract (#867) Fixed rebase on master SQL Query Engine (#878) * AggregationDataStore: Schema (#846) * AggregationDataStore: Static Attribute Aggregation * Address comments * Implement TimeDimension and all its supporting components * refactor * Address comments from @aklish * Address comments from @aklish && Tests & Javadoc * Address comments from @aklish * Address comments from @aklish and @hellohanchen * Address comments from Aaron * ToMany is not supported * Address comments from Aaron Added calcite as a dependency. Merged in changes for QueryEngine interface Fixed checkstyle issues Added basic H2 DB test harness Started breaking out projections Moved getValue and setValue from PersistentResource to EntityDictionary Added basic logic to hydrate entities Added FromTable and FromSubquery annotations. Add explicit exclusion of entity relationship hydration Minor cleanup Refactored HQLFilterOperation to take an alias generator Added test support for RSQL filter generation. Some cleanup Added basic support for WHERE clause filtering on the fact table Added working test for subquery SQL Added basic join logic for filters Added a test with a subquery and a filter join Refactored Schema classes and Query to support metric aggregation SQL expansion Added group by support Added logic for ID generation Added sorting logic and test Added pagination support and testing All column references use proper name now for SQL Removed calcite as a query engine Refactored HQLFilterOperation so it can be used for Having and Where clause generaiton Added HAVING clause support Changed Query to take schema instead of entityClass First pass at cleanup Fixed checkstyles Cleanup Cleanup Added a complex SQL expression test and fixed bugs Fixed merge issues. Added another test. Added better logging Fixed bug in pagination SQL generation * Build is working * Inspection rework Add EntityProjection plumbing (#949) * AggregationDataStore: Schema (#846) * AggregationDataStore: Static Attribute Aggregation * Address comments * Implement TimeDimension and all its supporting components * refactor * Address comments from @aklish * Address comments from @aklish && Tests & Javadoc * Address comments from @aklish * Address comments from @aklish and @hellohanchen * Address comments from Aaron * ToMany is not supported * Address comments from Aaron * Initial sketch PersistentResourceTest now passes LifeCycleTest tests now pass More API changes for data store transaction. Also fixed createObject in persistent resource to take the correct projection Started to refactor tests IncludedProcessorTest refactored Refactored LifeCycleTest Started refactor on PersistentResourceTest More refactoring. Fixed a bug in Resource.toPersistentResource Only one test failing in PersistentResourceTest PersistentResourceTests now pass UpdateOnCreateTests now pass Added skeleton for translating JSON-API URL path into an EntityProjection Basic EntityProjectionMaker almost complete Added ability to merge entity projections by relationship Added first test for EntityProjectionMaker Non-working veresion (but clean) Tests now pass All EntityProjectioNMaker tests pass Elide-Core now builds Added handling of sparse attributes and relationships Expanding attributes for included entities Fixed a number of bugs found in IT tests Fixed some of the EntityProjectionMaker tests Fixed unit tests Made temporary modifications to exclude GraphQL (Build now passes) Added sparse field unit tests for EntityProjectionMaker * Fixed build issues after rebase * Removed duplicated Schema class from rebase * Entity projection with aliases (#963) * Hacked up PersistentResource with new design * Core now compiles (and tests can run * EntityProjectionMaker tests pass * Build now passes (major cleanup still needed * Wire in entity projection4 json api (#964) * Fixed DataStore API. Fixed a lot of the core unit tests * Checkstyles and more fixes * Hibernate 5 Tests Pass * Full build passes * Wire in entity projection4 json api (#965) * Initial concept. No testing changed. * Core compiles and EntityProjectionMaker tests (original ones) now pass * Minor edits to TestRequestScope * Full build passes now * removed entity dictionary from entity projection * Pre-inspection cleanup * minor inspection fixup Hydrate Relationship (#987) * AggregationDataStore: Schema (#846) * AggregationDataStore: Static Attribute Aggregation * Address comments * Implement TimeDimension and all its supporting components * refactor * Address comments from @aklish * Address comments from @aklish && Tests & Javadoc * Address comments from @aklish * Address comments from @aklish and @hellohanchen * Address comments from Aaron * ToMany is not supported * Address comments from Aaron * Added basic H2 DB test harness * Started breaking out projections * Moved getValue and setValue from PersistentResource to EntityDictionary * Added basic logic to hydrate entities * Added FromTable and FromSubquery annotations. Add explicit exclusion of entity relationship hydration * Refactored HQLFilterOperation to take an alias generator * Added test support for RSQL filter generation. Some cleanup * Added basic support for WHERE clause filtering on the fact table * Added working test for subquery SQL * Added basic join logic for filters * Added a test with a subquery and a filter join * Refactored Schema classes and Query to support metric aggregation SQL expansion * Added group by support * Added logic for ID generation * Added sorting logic and test * Added pagination support and testing * All column references use proper name now for SQL * Removed calcite as a query engine * Refactored HQLFilterOperation so it can be used for Having and Where clause generaiton * Added HAVING clause support * Changed Query to take schema instead of entityClass * First pass at cleanup * Fixed checkstyles * Cleanup * Hydrate Relationship * Cleanup * Added a complex SQL expression test and fixed bugs * Fixed merge issues. Added another test. Added better logging * Self-review * Self-review * Self-review * Self-review * Self-review * Address comments from @aklish * Refactor EntityHydrator (#893) * rebase * keep Jiaqi's changes * fix id * fix maven verify * Remove HQLFilterOperation * fix dictionary * fix SqlEngineTest * remove unused part * make codacy happy * should use getParametrizedType * address comments Implement GraphQLEntityProjectionMaker (#986) * AggregationDataStore: Schema (#846) * AggregationDataStore: Static Attribute Aggregation * Address comments * Implement TimeDimension and all its supporting components * refactor * Address comments from @aklish * Address comments from @aklish && Tests & Javadoc * Address comments from @aklish * Address comments from @aklish and @hellohanchen * Address comments from Aaron * ToMany is not supported * Address comments from Aaron * Initial sketch PersistentResourceTest now passes LifeCycleTest tests now pass More API changes for data store transaction. Also fixed createObject in persistent resource to take the correct projection Started to refactor tests IncludedProcessorTest refactored Refactored LifeCycleTest Started refactor on PersistentResourceTest More refactoring. Fixed a bug in Resource.toPersistentResource Only one test failing in PersistentResourceTest PersistentResourceTests now pass UpdateOnCreateTests now pass Added skeleton for translating JSON-API URL path into an EntityProjection Basic EntityProjectionMaker almost complete Added ability to merge entity projections by relationship Added first test for EntityProjectionMaker Non-working veresion (but clean) Tests now pass All EntityProjectioNMaker tests pass Elide-Core now builds Added handling of sparse attributes and relationships Expanding attributes for included entities Fixed a number of bugs found in IT tests Fixed some of the EntityProjectionMaker tests Fixed unit tests Made temporary modifications to exclude GraphQL (Build now passes) Added sparse field unit tests for EntityProjectionMaker * Fixed build issues after rebase * GraphQL projection maker using document * Argument handling and fragment check * Add comments * Add fragment resolver * fix typo * break code into more methods * remove pagination and sorting * Removed duplicated Schema class from rebase * re-arrange keywords * Address comment * Add arguments for attribute fields * Handle arguments * support partial query, update edges/node logic * Entity projection with aliases * Entity projection with aliases (#963) * Hacked up PersistentResource with new design * Core now compiles (and tests can run * EntityProjectionMaker tests pass * Build now passes (major cleanup still needed * fix create relationship object using entity * Add tests passed * code clean up * refactor fatcher, fix test cases * rename keywords * rebase branch (#12) * rebased * Graphql projection refactor (#13) * fix fragment resolver * Fix variable resolver * Wire in entity projection4 json api (#964) * Fixed DataStore API. Fixed a lot of the core unit tests * Checkstyles and more fixes * Hibernate 5 Tests Pass * Full build passes * Wire in entity projection4 json api (#965) * Initial concept. No testing changed. * Core compiles and EntityProjectionMaker tests (original ones) now pass * Minor edits to TestRequestScope * Full build passes now * removed entity dictionary from entity projection * Pre-inspection cleanup * minor inspection fixup * rebase * Rebased on AggregationDataStore * clean up extra new lines * address comments * Builder pattern * update comments * remove projection in entity * fix jackson * Hydrate Relationship (#987) (#15) * Address some codecy comments * Add comment for partial query * Reenable tests * Address comments, refactor alias * Add test for alias * swapped test case * fix get type Added AggregationDataStore Code (#991) * Adding testing for aggregation data store * Debugging integration tests * Continuing testing work * AggregationDataStore * AggregationDataStore testing * Added more tests * Aggregation Data Store * Cleaned up testing code * Cleaned up code, fixed helper for AggregationDataStore * end * Fixed checkstyle, other minor fixes * fixed comment * Minor fixes * Fixed id type issue, added exception for queries with no metrics Fixed build (#993) Making TimeDimension an interface (#992) [maven-release-plugin] prepare release 5.0.0-pr1 [maven-release-plugin] prepare for next development iteration Renamed graphQL file to match test (#1002) [maven-release-plugin] prepare release 5.0.0-pr2 [maven-release-plugin] prepare for next development iteration Add JoinTo annotation (#1006) * Added JoinTo Annotation * Added working test * Added TODO comment for next PR * Added TODO comment for next PR * Added Sorting and Filtering support for JoinTo Columns * Fixed IT tests for Aggregation Data Store * Moved entityManager creation to happen separately for each query (#1008) * Moved entityManager creation to happen separately for each query * Closing EntityManager after each query * Inspection rework Column annotation (#1017) * Solved column issue and added QueryEngineFactory * Caching query engine in AggregationDataStore * Fixed column description * Update SQLQueryEngine.java (#1019) * Add SQLMetrix, rearrange packages (#1020) * Add SQLMetrix, rearrange packages * address comment Manager transacton manually (#1021) * Manager transacton manually * Add readonly Hydrate GraphQL Schema with parameterized attributes (#1018) * GraphQL schema expose expected argument name and its type for each attribute * Change empty arguments to unmodifable set AggregationStore: Add multiple time grain definitions to schema (#1022) * Fixed checkstyle warnings and errors. Separated the Query dimension interface from the Schema dimension interface * Added skeleton code to convert entity projection arguments into time grains * Cleanup * Class renames per inspection comments * Inspection comments Refactor time dimension logic (#1028) * Manager transacton manually * Add readonly * some rework * use getTimeDimension() * change exception ISSUE-1026 Add support for @Subselect (#1038) * Manager transacton manually * Add readonly * some rework * use getTimeDimension() * change exception * ISSUE-1026 Add support for @Subselect * Address comments ISSUE-1027 Support join for having clause (#1039) * Manager transacton manually * Add readonly * some rework * use getTimeDimension() * change exception * ISSUE-1027 Support join for having clause function name fixed to enableISO8601Dates (#1052) Support for multiple queries at root is added (#1044) * Support for multiple queries at root is added * Added test with alias * comments resolved Add time grain to GraphQL schema (#1042) * Added basic plumbing to push attributes from the entity projection down to the QueryEngine * Added logic to expand SQL time expression in SQLQueryEngine * Added SQLQueryEngine tests * Added IT tests * The AggregationStore now adds graphql parameters for parameterized columns * Minor refactor * Inspection rework * Minor fix Support multiple query of same entity with different alias (#1055) * Support multiple query of same entity with different alias * add static method to generate keyname for GraphQLProjectionInfo projections * Remove aliasPartialQuerySameAttribute MetadataStore Models (#1068) * Manager transacton manually * Add readonly * some rework * use getTimeDimension() * change exception * Metadatastore models * Address comments * address comments * move root * fix style check SQLQueryTemplate Model (#1073) * SQLQueryTemplate * SQLTables * refactor * update sql dimension projection * update sql dimension projection * clean up dimension projection * refactor sql components * aggregatable field rework * add comments * rearrange packages * Add dimension projection back * fix checkstyle * Add dictionary * Simplify MetricFunction and SQLQueryTemplate * Address comments Integrate Metadata Model and SQLQueryTemplate Model (#1083) * Integrate Metadata Model and SQLQueryTemplate Model * remove AggregationDictionary and AggregationManager * Add timezone * Can only query analyticView Fixed issues with rebase Add auto configuration for aggregation store (#1087) * Added autoconfiguration for QueryEngineFactory * Unified class scanning. Started cleaning up datastores so they only register the entities they manage * Full build passes * Minor cleanup * Minor refactoring * Added EntityManagerFactory bean configuratino * Refactored class scanning for Elide standalone * Updated spring boot starter pom * Removed @Entity from all metadata models. Started cleaning up entity dictionary entity registration * Broken implementation. Just checking in so I can revert if needed. * All tests pass * Added unit tests * Minor cleanup * One more fix * Fixed broken tests * Added package include support back * Class scanning for annotations ignores inherited * Added a test based on inspection comments * Inspection comment fix * Changed initalization of MetadataStore * More inspection rework * Turned back on OWASP scanning * More rework remove @Inherited (#1092) Support Non JPA Entity in AggregationDataStore (#1051) * Create AggregationDataStore module (#845) * Create AggregationDataStore module * Address Aaron's comments * Fix build failure AggregationDataStore: Schema (#846) * AggregationDataStore: Static Attribute Aggregation * Address comments * Implement TimeDimension and all its supporting components * refactor * Address comments from @aklish * Address comments from @aklish && Tests & Javadoc * Address comments from @aklish * Address comments from @aklish and @hellohanchen * Address comments from Aaron * ToMany is not supported * Address comments from Aaron Define QueryEngine Contract (#867) Fixed rebase on master SQL Query Engine (#878) * AggregationDataStore: Schema (#846) * AggregationDataStore: Static Attribute Aggregation * Address comments * Implement TimeDimension and all its supporting components * refactor * Address comments from @aklish * Address comments from @aklish && Tests & Javadoc * Address comments from @aklish * Address comments from @aklish and @hellohanchen * Address comments from Aaron * ToMany is not supported * Address comments from Aaron Added calcite as a dependency. Merged in changes for QueryEngine interface Fixed checkstyle issues Added basic H2 DB test harness Started breaking out projections Moved getValue and setValue from PersistentResource to EntityDictionary Added basic logic to hydrate entities Added FromTable and FromSubquery annotations. Add explicit exclusion of entity relationship hydration Minor cleanup Refactored HQLFilterOperation to take an alias generator Added test support for RSQL filter generation. Some cleanup Added basic support for WHERE clause filtering on the fact table Added working test for subquery SQL Added basic join logic for filters Added a test with a subquery and a filter join Refactored Schema classes and Query to support metric aggregation SQL expansion Added group by support Added logic for ID generation Added sorting logic and test Added pagination support and testing All column references use proper name now for SQL Removed calcite as a query engine Refactored HQLFilterOperation so it can be used for Having and Where clause generaiton Added HAVING clause support Changed Query to take schema instead of entityClass First pass at cleanup Fixed checkstyles Cleanup Cleanup Added a complex SQL expression test and fixed bugs Fixed merge issues. Added another test. Added better logging Fixed bug in pagination SQL generation * Build is working * Inspection rework Add EntityProjection plumbing (#949) * AggregationDataStore: Schema (#846) * AggregationDataStore: Static Attribute Aggregation * Address comments * Implement TimeDimension and all its supporting components * refactor * Address comments from @aklish * Address comments from @aklish && Tests & Javadoc * Address comments from @aklish * Address comments from @aklish and @hellohanchen * Address comments from Aaron * ToMany is not supported * Address comments from Aaron * Initial sketch PersistentResourceTest now passes LifeCycleTest tests now pass More API changes for data store transaction. Also fixed createObject in persistent resource to take the correct projection Started to refactor tests IncludedProcessorTest refactored Refactored LifeCycleTest Started refactor on PersistentResourceTest More refactoring. Fixed a bug in Resource.toPersistentResource Only one test failing in PersistentResourceTest PersistentResourceTests now pass UpdateOnCreateTests now pass Added skeleton for translating JSON-API URL path into an EntityProjection Basic EntityProjectionMaker almost complete Added ability to merge entity projections by relationship Added first test for EntityProjectionMaker Non-working veresion (but clean) Tests now pass All EntityProjectioNMaker tests pass Elide-Core now builds Added handling of sparse attributes and relationships Expanding attributes for included entities Fixed a number of bugs found in IT tests Fixed some of the EntityProjectionMaker tests Fixed unit tests Made temporary modifications to exclude GraphQL (Build now passes) Added sparse field unit tests for EntityProjectionMaker * Fixed build issues after rebase * Removed duplicated Schema class from rebase * Entity projection with aliases (#963) * Hacked up PersistentResource with new design * Core now compiles (and tests can run * EntityProjectionMaker tests pass * Build now passes (major cleanup still needed * Wire in entity projection4 json api (#964) * Fixed DataStore API. Fixed a lot of the core unit tests * Checkstyles and more fixes * Hibernate 5 Tests Pass * Full build passes * Wire in entity projection4 json api (#965) * Initial concept. No testing changed. * Core compiles and EntityProjectionMaker tests (original ones) now pass * Minor edits to TestRequestScope * Full build passes now * removed entity dictionary from entity projection * Pre-inspection cleanup * minor inspection fixup Hydrate Relationship (#987) * AggregationDataStore: Schema (#846) * AggregationDataStore: Static Attribute Aggregation * Address comments * Implement TimeDimension and all its supporting components * refactor * Address comments from @aklish * Address comments from @aklish && Tests & Javadoc * Address comments from @aklish * Address comments from @aklish and @hellohanchen * Address comments from Aaron * ToMany is not supported * Address comments from Aaron * Added basic H2 DB test harness * Started breaking out projections * Moved getValue and setValue from PersistentResource to EntityDictionary * Added basic logic to hydrate entities * Added FromTable and FromSubquery annotations. Add explicit exclusion of entity relationship hydration * Refactored HQLFilterOperation to take an alias generator * Added test support for RSQL filter generation. Some cleanup * Added basic support for WHERE clause filtering on the fact table * Added working test for subquery SQL * Added basic join logic for filters * Added a test with a subquery and a filter join * Refactored Schema classes and Query to support metric aggregation SQL expansion * Added group by support * Added logic for ID generation * Added sorting logic and test * Added pagination support and testing * All column references use proper name now for SQL * Removed calcite as a query engine * Refactored HQLFilterOperation so it can be used for Having and Where clause generaiton * Added HAVING clause support * Changed Query to take schema instead of entityClass * First pass at cleanup * Fixed checkstyles * Cleanup * Hydrate Relationship * Cleanup * Added a complex SQL expression test and fixed bugs * Fixed merge issues. Added another test. Added better logging * Self-review * Self-review * Self-review * Self-review * Self-review * Address comments from @aklish * Refactor EntityHydrator (#893) * rebase * keep Jiaqi's changes * fix id * fix maven verify * Remove HQLFilterOperation * fix dictionary * fix SqlEngineTest * remove unused part * make codacy happy * should use getParametrizedType * address comments Implement GraphQLEntityProjectionMaker (#986) * AggregationDataStore: Schema (#846) * AggregationDataStore: Static Attribute Aggregation * Address comments * Implement TimeDimension and all its supporting components * refactor * Address comments from @aklish * Address comments from @aklish && Tests & Javadoc * Address comments from @aklish * Address comments from @aklish and @hellohanchen * Address comments from Aaron * ToMany is not supported * Address comments from Aaron * Initial sketch PersistentResourceTest now passes LifeCycleTest tests now pass More API changes for data store transaction. Also fixed createObject in persistent resource to take the correct projection Started to refactor tests IncludedProcessorTest refactored Refactored LifeCycleTest Started refactor on PersistentResourceTest More refactoring. Fixed a bug in Resource.toPersistentResource Only one test failing in PersistentResourceTest PersistentResourceTests now pass UpdateOnCreateTests now pass Added skeleton for translating JSON-API URL path into an EntityProjection Basic EntityProjectionMaker almost complete Added ability to merge entity projections by relationship Added first test for EntityProjectionMaker Non-working veresion (but clean) Tests now pass All EntityProjectioNMaker tests pass Elide-Core now builds Added handling of sparse attributes and relationships Expanding attributes for included entities Fixed a number of bugs found in IT tests Fixed some of the EntityProjectionMaker tests Fixed unit tests Made temporary modifications to exclude GraphQL (Build now passes) Added sparse field unit tests for EntityProjectionMaker * Fixed build issues after rebase * GraphQL projection maker using document * Argument handling and fragment check * Add comments * Add fragment resolver * fix typo * break code into more methods * remove pagination and sorting * Removed duplicated Schema class from rebase * re-arrange keywords * Address comment * Add arguments for attribute fields * Handle arguments * support partial query, update edges/node logic * Entity projection with aliases * Entity projection with aliases (#963) * Hacked up PersistentResource with new design * Core now compiles (and tests can run * EntityProjectionMaker tests pass * Build now passes (major cleanup still needed * fix create relationship object using entity * Add tests passed * code clean up * refactor fatcher, fix test cases * rename keywords * rebase branch (#12) * rebased * Graphql projection refactor (#13) * fix fragment resolver * Fix variable resolver * Wire in entity projection4 json api (#964) * Fixed DataStore API. Fixed a lot of the core unit tests * Checkstyles and more fixes * Hibernate 5 Tests Pass * Full build passes * Wire in entity projection4 json api (#965) * Initial concept. No testing changed. * Core compiles and EntityProjectionMaker tests (original ones) now pass * Minor edits to TestRequestScope * Full build passes now * removed entity dictionary from entity projection * Pre-inspection cleanup * minor inspection fixup * rebase * Rebased on AggregationDataStore * clean up extra new lines * address comments * Builder pattern * update comments * remove projection in entity * fix jackson * Hydrate Relationship (#987) (#15) * Address some codecy comments * Add comment for partial query * Reenable tests * Address comments, refactor alias * Add test for alias * swapped test case * fix get type Added AggregationDataStore Code (#991) * Adding testing for aggregation data store * Debugging integration tests * Continuing testing work * AggregationDataStore * AggregationDataStore testing * Added more tests * Aggregation Data Store * Cleaned up testing code * Cleaned up code, fixed helper for AggregationDataStore * end * Fixed checkstyle, other minor fixes * fixed comment * Minor fixes * Fixed id type issue, added exception for queries with no metrics Fixed build (#993) Making TimeDimension an interface (#992) * [maven-release-plugin] prepare release 5.0.0-pr1 * [maven-release-plugin] prepare for next development iteration * Renamed graphQL file to match test (#1002) * [maven-release-plugin] prepare release 5.0.0-pr2 * [maven-release-plugin] prepare for next development iteration * Add JoinTo annotation (#1006) * Added JoinTo Annotation * Added working test * Added TODO comment for next PR * Added TODO comment for next PR * Added Sorting and Filtering support for JoinTo Columns * Fixed IT tests for Aggregation Data Store * Moved entityManager creation to happen separately for each query (#1008) * Moved entityManager creation to happen separately for each query * Closing EntityManager after each query * Inspection rework * Column annotation (#1017) * Solved column issue and added QueryEngineFactory * Caching query engine in AggregationDataStore * Fixed column description * Update SQLQueryEngine.java (#1019) * Add SQLMetrix, rearrange packages (#1020) * Add SQLMetrix, rearrange packages * address comment * Manager transacton manually * Add readonly * Manager transacton manually (#1021) * Manager transacton manually * Add readonly * Hydrate GraphQL Schema with parameterized attributes (#1018) * GraphQL schema expose expected argument name and its type for each attribute * Change empty arguments to unmodifable set * AggregationStore: Add multiple time grain definitions to schema (#1022) * Fixed checkstyle warnings and errors. Separated the Query dimension interface from the Schema dimension interface * Added skeleton code to convert entity projection arguments into time grains * Cleanup * Class renames per inspection comments * Inspection comments * some rework * use getTimeDimension() * change exception * Refactor time dimension logic (#1028) * Manager transacton manually * Add readonly * some rework * use getTimeDimension() * change exception * ISSUE-1026 Add support for @Subselect (#1038) * Manager transacton manually * Add readonly * some rework * use getTimeDimension() * change exception * ISSUE-1026 Add support for @Subselect * Address comments * ISSUE-1027 Support join for having clause (#1039) * Manager transacton manually * Add readonly * some rework * use getTimeDimension() * change exception * ISSUE-1027 Support join for having clause * View Design * Add tests and cleanup * rename annotation * function name fixed to enableISO8601Dates (#1052) * fix bugs * Support for multiple queries at root is added (#1044) * Support for multiple queries at root is added * Added test with alias * comments resolved * merge annotations * don't group by view relationship * Add time grain to GraphQL schema (#1042) * Added basic plumbing to push attributes from the entity projection down to the QueryEngine * Added logic to expand SQL time expression in SQLQueryEngine * Added SQLQueryEngine tests * Added IT tests * The AggregationStore now adds graphql parameters for parameterized columns * Minor refactor * Inspection rework * Minor fix * Support multiple query of same entity with different alias (#1055) * Support multiple query of same entity with different alias * add static method to generate keyname for GraphQLProjectionInfo projections * Remove aliasPartialQuerySameAttribute * MetadataStore Models (#1068) * Manager transacton manually * Add readonly * some rework * use getTimeDimension() * change exception * Metadatastore models * Address comments * address comments * move root * fix style check * SQLQueryTemplate Model (#1073) * SQLQueryTemplate * SQLTables * refactor * update sql dimension projection * update sql dimension projection * clean up dimension projection * refactor sql components * aggregatable field rework * add comments * rearrange packages * Add dimension projection back * fix checkstyle * Add dictionary * Simplify MetricFunction and SQLQueryTemplate * Address comments * Integrate Metadata Model and SQLQueryTemplate Model (#1083) * Integrate Metadata Model and SQLQueryTemplate Model * remove AggregationDictionary and AggregationManager * Add timezone * Can only query analyticView * integrate view with aggregation and metadata * remove includeField * remove @view * Use NonEntityDictionary * remove id * revert access changes * fix JPA entity check * remove @Entity from analyticViews * use table name as relationship type id * revert NonEntitydictinoary * tiny rework * Integration tests * Add jsonapi ittest * aggregation data store doesn't manage jpa entities * address comments fix integration dependencies (#1093) [maven-release-plugin] prepare release 5.0.0-pr3 [maven-release-plugin] prepare for next development iteration Fixed elide standalone pom from rebase Fixed minor bug in rebase Fixed rebase Improving class scanning performance for MetadataStore (#1117) Enable elide5 travis builds (#1129) * Move repeated @Sql annotations to class level (#1119) * Turning on travis builds with code coverage for Elide 5.x * Fixing security issue in spring-boot-web Co-authored-by: Brutus5000 Fix sorting and ambiguous join issue (#1127) * Added sorting on aggregated metric based on latest elide-5.x * Fix ambiguity problem * update comments * fix codacy * refactor generateColumnReference * update comment * address comments * test cleanup * update unittest * fix elide core alias * QueryValidatorTest * EntityProjectionTranslatorTest * go joinFragment approach * delete jointrienode Support no metric query (#1137) [maven-release-plugin] prepare release 5.0.0-pr4 [maven-release-plugin] prepare for next development iteration Check dependency injection (#1138) * Move repeated @Sql annotations to class level (#1119) * Fixing OWASP security warning for Tomcat dependency in Spring Web (#1132) * Adding support for dependency injection of Checks. Added test injection classes * Unit tests pass * Tests pass * Removed Initializer Concept Co-authored-by: Brutus5000 Fix travis log length (#1140) * Move repeated @Sql annotations to class level (#1119) * Fixing OWASP security warning for Tomcat dependency in Spring Web (#1132) * Removed unnecessary request/response logging (to shorten travis logs) * Address inspection comments * Address inspection comments * Address inspection comments * Removed logging of graphQL model building to shorten length * Fixed compilation error Co-authored-by: Brutus5000 [maven-release-plugin] prepare release 5.0.0-pr5 [maven-release-plugin] prepare for next development iteration --- .travis.yml | 1 + checkstyle-suppressions.xml | 2 +- elide-annotations/pom.xml | 3 +- .../com/yahoo/elide/annotation/Exclude.java | 2 - .../com/yahoo/elide/annotation/Include.java | 2 - elide-contrib/elide-swagger/pom.xml | 6 +- elide-contrib/elide-test-helpers/pom.xml | 2 +- .../testhelpers/graphql/GraphQLDSL.java | 4 + .../testhelpers/graphql/elements/Field.java | 2 +- elide-contrib/pom.xml | 4 +- elide-core/pom.xml | 10 +- .../src/main/java/com/yahoo/elide/Elide.java | 9 + .../com/yahoo/elide/core/ArgumentType.java | 23 + .../yahoo/elide/core/CheckInstantiator.java | 9 +- .../elide/core/DataStoreTransaction.java | 78 +- .../com/yahoo/elide/core/EntityBinding.java | 40 +- .../yahoo/elide/core/EntityDictionary.java | 171 +++- .../com/yahoo/elide/core/Initializer.java | 22 - .../yahoo/elide/core/PersistentResource.java | 349 ++++--- .../com/yahoo/elide/core/RequestScope.java | 28 +- .../com/yahoo/elide/core/TimedFunction.java | 40 + .../datastore/inmemory/HashMapDataStore.java | 4 +- .../inmemory/HashMapStoreTransaction.java | 24 +- .../inmemory/InMemoryStoreTransaction.java | 159 +-- .../datastore/wrapped/TransactionWrapper.java | 33 +- .../elide/core/filter/FilterPredicate.java | 33 +- .../filter/dialect/RSQLFilterDialect.java | 1 - .../expression/AndFilterExpression.java | 26 + .../filter/expression/OrFilterExpression.java | 26 + .../elide/core/pagination/Pagination.java | 19 +- .../com/yahoo/elide/core/sort/Sorting.java | 2 + .../elide/extensions/PatchRequestScope.java | 2 + .../elide/jsonapi/EntityProjectionMaker.java | 383 +++++++ .../processors/IncludedProcessor.java | 20 +- .../yahoo/elide/jsonapi/models/Resource.java | 13 +- .../jsonapi/models/ResourceIdentifier.java | 5 +- .../state/CollectionTerminalState.java | 30 +- .../elide/parsers/state/RecordState.java | 116 +-- .../state/RelationshipTerminalState.java | 11 +- .../yahoo/elide/parsers/state/StartState.java | 36 +- .../com/yahoo/elide/request/Argument.java | 32 + .../com/yahoo/elide/request/Attribute.java | 43 + .../yahoo/elide/request/EntityProjection.java | 261 +++++ .../com/yahoo/elide/request/Relationship.java | 48 + .../elide/security/FilterExpressionCheck.java | 28 +- .../com/yahoo/elide/utils/ClassScanner.java | 18 +- .../elide/core/EntityDictionaryTest.java | 85 +- .../com/yahoo/elide/core/LifeCycleTest.java | 70 +- .../elide/core/PermissionAnnotationTest.java | 3 +- .../core/PersistenceResourceTestSetup.java | 4 +- .../elide/core/PersistentResourceTest.java | 774 ++++++++++---- .../com/yahoo/elide/core/TestDictionary.java | 70 ++ .../com/yahoo/elide/core/TestInjector.java | 33 + .../yahoo/elide/core/TestRequestScope.java | 53 + .../yahoo/elide/core/UpdateOnCreateTest.java | 365 +++++-- .../InMemoryStoreTransactionTest.java | 308 +++--- .../wrapped/TransactionWrapperTest.java | 35 +- .../elide/core/utils/ClassScannerTest.java | 11 + .../jsonapi/EntityProjectionMakerTest.java | 745 ++++++++++++++ .../com/yahoo/elide/jsonapi/JsonApiTest.java | 65 +- .../processors/IncludedProcessorTest.java | 59 +- .../expression/CanPaginateVisitorTest.java | 27 +- .../PermissionExpressionVisitorTest.java | 3 +- ...rmissionToFilterExpressionVisitorTest.java | 3 +- .../security/PermissionExecutorTest.java | 6 +- .../PermissionExpressionBuilderTest.java | 3 +- .../elide-datastore-aggregation/.gitignore | 1 + .../elide-datastore-aggregation/pom.xml | 203 ++++ .../aggregation/AggregationDataStore.java | 72 ++ .../AggregationDataStoreTransaction.java | 71 ++ .../EntityProjectionTranslator.java | 206 ++++ .../datastores/aggregation/QueryEngine.java | 72 ++ .../aggregation/QueryEngineFactory.java | 15 + .../aggregation/QueryValidator.java | 155 +++ .../aggregation/annotation/Cardinality.java | 44 + .../annotation/CardinalitySize.java | 34 + .../aggregation/annotation/FriendlyName.java | 23 + .../aggregation/annotation/Meta.java | 25 + .../annotation/MetricAggregation.java | 24 + .../annotation/MetricComputation.java | 62 ++ .../aggregation/annotation/Temporal.java | 47 + .../annotation/TimeGrainDefinition.java | 29 + .../filter/visitor/FilterConstraints.java | 111 ++ .../visitor/SplitFilterExpressionVisitor.java | 238 +++++ .../aggregation/metadata/MetaDataStore.java | 183 ++++ .../metadata/enums/Aggregation.java | 15 + .../aggregation/metadata/enums/Format.java | 14 + .../aggregation/metadata/enums/Tag.java | 13 + .../aggregation/metadata/enums/ValueType.java | 19 + .../metric/MetricFunctionInvocation.java | 69 ++ .../metadata/models/AnalyticView.java | 48 + .../aggregation/metadata/models/Column.java | 87 ++ .../aggregation/metadata/models/DataType.java | 58 ++ .../metadata/models/Dimension.java | 24 + .../metadata/models/FunctionArgument.java | 39 + .../aggregation/metadata/models/Metric.java | 50 + .../metadata/models/MetricFunction.java | 114 +++ .../metadata/models/RelationshipType.java | 22 + .../aggregation/metadata/models/Table.java | 148 +++ .../metadata/models/TimeDimension.java | 44 + .../metadata/models/TimeDimensionGrain.java | 35 + .../aggregation/query/ColumnProjection.java | 91 ++ .../datastores/aggregation/query/Query.java | 57 ++ .../query/TimeDimensionProjection.java | 86 ++ .../queryengines/AbstractEntityHydrator.java | 203 ++++ .../aggregation/queryengines/StitchList.java | 162 +++ .../queryengines/sql/SQLEntityHydrator.java | 83 ++ .../queryengines/sql/SQLQuery.java | 53 + .../queryengines/sql/SQLQueryConstructor.java | 546 ++++++++++ .../queryengines/sql/SQLQueryEngine.java | 299 ++++++ .../sql/SQLQueryEngineFactory.java | 30 + .../sql/annotation/FromSubquery.java | 28 + .../sql/annotation/FromTable.java | 28 + .../queryengines/sql/annotation/JoinTo.java | 37 + .../sql/metadata/SQLAnalyticView.java | 34 + .../queryengines/sql/metadata/SQLColumn.java | 42 + .../queryengines/sql/metadata/SQLTable.java | 51 + .../sql/metric/SQLMetricFunction.java | 62 ++ .../sql/metric/functions/SqlAvg.java | 19 + .../sql/metric/functions/SqlMax.java | 19 + .../sql/metric/functions/SqlMin.java | 19 + .../sql/metric/functions/SqlSum.java | 19 + .../sql/query/SQLQueryTemplate.java | 111 ++ .../aggregation/time/TimeGrain.java | 26 + .../EntityProjectionTranslatorTest.java | 164 +++ .../aggregation/QueryValidatorTest.java | 204 ++++ .../aggregation/example/Continent.java | 28 + .../aggregation/example/Country.java | 73 ++ .../aggregation/example/CountryView.java | 80 ++ .../example/CountryViewNested.java | 54 + .../aggregation/example/Player.java | 34 + .../aggregation/example/PlayerStats.java | 235 +++++ .../aggregation/example/PlayerStatsView.java | 46 + .../example/PlayerStatsWithView.java | 223 +++++ .../aggregation/example/SubCountry.java | 61 ++ .../aggregation/example/VideoGame.java | 79 ++ .../filter/visitor/FilterConstraintsTest.java | 70 ++ .../SplitFilterExpressionVisitorTest.java | 176 ++++ .../AggregationDataStoreTestHarness.java | 42 + .../aggregation/framework/SQLUnitTest.java | 99 ++ .../AggregationDataStoreIntegrationTest.java | 947 ++++++++++++++++++ .../metadata/MetaDataStoreTest.java | 47 + .../queryengines/sql/QueryEngineTest.java | 681 +++++++++++++ .../queryengines/sql/SubselectTest.java | 267 +++++ .../queryengines/sql/ViewTest.java | 261 +++++ .../test/resources/META-INF/persistence.xml | 28 + .../src/test/resources/continent.csv | 3 + .../src/test/resources/country.csv | 3 + .../src/test/resources/create_tables.sql | 37 + .../src/test/resources/player.csv | 4 + .../src/test/resources/player_stats.csv | 4 + .../src/test/resources/video_games.csv | 5 + .../elide-datastore-hibernate/pom.xml | 4 +- .../elide/core/filter/FilterTranslator.java | 57 +- .../hql/AbstractHQLQueryBuilder.java | 21 +- .../hql/RootCollectionFetchQueryBuilder.java | 2 +- .../RootCollectionPageTotalsQueryBuilder.java | 4 +- .../hql/SubCollectionFetchQueryBuilder.java | 2 +- .../SubCollectionPageTotalsQueryBuilder.java | 2 +- .../core/filter/FilterTranslatorTest.java | 3 +- .../hql/AbstractHQLQueryBuilderTest.java | 4 +- .../RootCollectionFetchQueryBuilderTest.java | 9 + ...tCollectionPageTotalsQueryBuilderTest.java | 19 +- .../SubCollectionFetchQueryBuilderTest.java | 2 + ...bCollectionPageTotalsQueryBuilderTest.java | 18 +- .../elide-datastore-hibernate3/pom.xml | 8 +- .../hibernate3/HibernateTransaction.java | 76 +- .../elide-datastore-hibernate5/pom.xml | 8 +- .../hibernate5/HibernateTransaction.java | 78 +- .../elide-datastore-inmemorydb/pom.xml | 4 +- .../inmemory/HashMapDataStoreTest.java | 22 +- elide-datastore/elide-datastore-jpa/pom.xml | 6 +- .../transaction/AbstractJpaTransaction.java | 76 +- .../elide-datastore-multiplex/pom.xml | 8 +- .../multiplex/MultiplexManager.java | 15 +- .../multiplex/MultiplexTransaction.java | 61 +- .../multiplex/MultiplexWriteTransaction.java | 30 +- .../multiplex/MultiplexManagerTest.java | 33 +- .../datastores/multiplex/TestDataStore.java | 17 +- .../bridgeable/BridgeableRedisStore.java | 64 +- elide-datastore/elide-datastore-noop/pom.xml | 2 +- .../datastores/noop/NoopTransaction.java | 28 +- .../java/com/yahoo/elide/beans/NoopBean.java | 2 +- .../datastores/noop/NoopTransactionTest.java | 21 +- .../elide-datastore-search/pom.xml | 6 +- .../search/SearchDataTransaction.java | 22 +- .../datastores/search/DataStoreLoadTest.java | 114 ++- elide-datastore/pom.xml | 5 +- elide-example-models/pom.xml | 2 +- .../elide-blog-example-resteasy/pom.xml | 6 +- elide-example/elide-blog-example/pom.xml | 8 +- .../elide-hibernate3-mysql-example/pom.xml | 6 +- elide-example/pom.xml | 4 +- elide-graphql/pom.xml | 6 +- .../java/com/yahoo/elide/graphql/Entity.java | 67 +- .../com/yahoo/elide/graphql/Environment.java | 2 +- .../elide/graphql/GraphQLConversionUtils.java | 40 + .../yahoo/elide/graphql/GraphQLEndpoint.java | 2 - .../elide/graphql/GraphQLRequestScope.java | 19 +- .../java/com/yahoo/elide/graphql/KeyWord.java | 47 + .../com/yahoo/elide/graphql/ModelBuilder.java | 13 +- .../graphql/PersistentResourceFetcher.java | 208 ++-- .../com/yahoo/elide/graphql/QueryRunner.java | 25 +- .../containers/ConnectionContainer.java | 10 +- .../graphql/containers/EdgesContainer.java | 6 +- .../graphql/containers/NodeContainer.java | 24 +- .../graphql/containers/PageInfoContainer.java | 19 +- .../graphql/containers/RootContainer.java | 22 +- .../graphql/parser/FragmentResolver.java | 139 +++ .../parser/GraphQLEntityProjectionMaker.java | 557 ++++++++++ .../graphql/parser/GraphQLProjectionInfo.java | 37 + .../graphql/parser/VariableResolver.java | 122 +++ .../elide/graphql/FetcherDeleteTest.java | 11 + .../yahoo/elide/graphql/FetcherFetchTest.java | 106 +- .../elide/graphql/FetcherRemoveTest.java | 12 + .../elide/graphql/FetcherReplaceTest.java | 7 +- .../elide/graphql/GraphQLEndpointTest.java | 241 ++++- .../com/yahoo/elide/graphql/GraphQLTest.java | 2 +- .../yahoo/elide/graphql/ModelBuilderTest.java | 22 + .../PersistentResourceFetcherTest.java | 34 +- .../graphqlEndpointTestModels/Author.java | 2 +- .../requests/fetch/aliasAmbiguous.graphql | 10 + .../requests/fetch/aliasAttribute.graphql | 9 + .../fetch/aliasPartialQueryAmbiguous.graphql | 23 + .../requests/fetch/aliasRelationship.graphql | 18 + .../fetch/aliasSameRelationship.graphql | 26 + ...agment.graphql => fragmentCorrect.graphql} | 0 .../requests/fetch/fragmentInline.graphql | 26 + .../requests/fetch/fragmentLoop.graphql | 31 + .../requests/fetch/fragmentUnknown.graphql | 31 + .../fetch/nestedCollectionFilter.graphql | 4 +- .../fetch/rootCollectionInvalidSort.graphql | 10 + .../requests/fetch/rootMultiple.graphql | 16 + .../requests/fetch/rootUnknownField.graphql | 18 + .../requests/fetch/variableDefinition.graphql | 18 + .../fetch/variableInvalidNonNull.graphql | 18 + .../fetch/variableUnknownReference.graphql | 16 + ...ls.graphql => replaceWithIdsFails.graphql} | 0 .../responses/fetch/aliasAttribute.json | 11 + .../responses/fetch/aliasRelationship.json | 22 + .../fetch/aliasSameRelationship.json | 32 + ...WithFragment.json => fragmentCorrect.json} | 0 .../responses/fetch/variableDefinition.json | 22 + elide-integration-tests/pom.xml | 4 +- .../EncodedErrorObjectsIT.java | 2 +- .../EncodedErrorResponsesIT.java | 2 +- .../VerboseEncodedErrorResponsesIT.java | 14 - .../AbstractApiResourceInitializer.java | 7 +- .../EncodedErrorResponsesTestBinder.java | 5 +- ...egrationTestApplicationResourceConfig.java | 9 +- .../ErrorObjectsTestBinder.java | 10 +- .../elide/initialization/IntegrationTest.java | 2 +- .../initialization/StandardTestBinder.java | 7 +- .../java/com/yahoo/elide/tests/GraphQLIT.java | 145 --- .../com/yahoo/elide/tests/ResourceIT.java | 11 +- .../graphQLFetchError.json | 10 - .../graphQLFetchErrorObjectEncoded.json | 7 + .../graphQLFetchErrorResponseEncoded.json | 5 + .../elide-spring-boot-autoconfigure/pom.xml | 23 +- .../spring/config/ElideAutoConfiguration.java | 28 +- .../spring/models/aggregation/Stats.java | 42 + .../models/{ => jpa}/ArtifactGroup.java | 2 +- .../models/{ => jpa}/ArtifactProduct.java | 2 +- .../models/{ => jpa}/ArtifactVersion.java | 2 +- .../spring/tests/AggregationStoreTest.java | 54 + .../elide/spring/tests/ControllerTest.java | 3 +- .../src/test/resources/application.yaml | 2 +- .../elide-spring-boot-starter/pom.xml | 40 +- elide-spring/pom.xml | 2 +- elide-standalone/pom.xml | 12 +- .../elide/standalone/ElideStandalone.java | 4 +- .../config/ElideStandaloneSettings.java | 10 +- pom.xml | 6 +- 273 files changed, 14877 insertions(+), 2013 deletions(-) create mode 100644 elide-core/src/main/java/com/yahoo/elide/core/ArgumentType.java delete mode 100644 elide-core/src/main/java/com/yahoo/elide/core/Initializer.java create mode 100644 elide-core/src/main/java/com/yahoo/elide/core/TimedFunction.java create mode 100644 elide-core/src/main/java/com/yahoo/elide/jsonapi/EntityProjectionMaker.java create mode 100644 elide-core/src/main/java/com/yahoo/elide/request/Argument.java create mode 100644 elide-core/src/main/java/com/yahoo/elide/request/Attribute.java create mode 100644 elide-core/src/main/java/com/yahoo/elide/request/EntityProjection.java create mode 100644 elide-core/src/main/java/com/yahoo/elide/request/Relationship.java create mode 100644 elide-core/src/test/java/com/yahoo/elide/core/TestDictionary.java create mode 100644 elide-core/src/test/java/com/yahoo/elide/core/TestInjector.java create mode 100644 elide-core/src/test/java/com/yahoo/elide/core/TestRequestScope.java create mode 100644 elide-core/src/test/java/com/yahoo/elide/jsonapi/EntityProjectionMakerTest.java create mode 100644 elide-datastore/elide-datastore-aggregation/.gitignore create mode 100644 elide-datastore/elide-datastore-aggregation/pom.xml create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/AggregationDataStore.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/AggregationDataStoreTransaction.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/EntityProjectionTranslator.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/QueryEngine.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/QueryEngineFactory.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/QueryValidator.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/annotation/Cardinality.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/annotation/CardinalitySize.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/annotation/FriendlyName.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/annotation/Meta.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/annotation/MetricAggregation.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/annotation/MetricComputation.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/annotation/Temporal.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/annotation/TimeGrainDefinition.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/filter/visitor/FilterConstraints.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/filter/visitor/SplitFilterExpressionVisitor.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/MetaDataStore.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/enums/Aggregation.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/enums/Format.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/enums/Tag.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/enums/ValueType.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/metric/MetricFunctionInvocation.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/AnalyticView.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/Column.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/DataType.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/Dimension.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/FunctionArgument.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/Metric.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/MetricFunction.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/RelationshipType.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/Table.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/TimeDimension.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/TimeDimensionGrain.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/query/ColumnProjection.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/query/Query.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/query/TimeDimensionProjection.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/AbstractEntityHydrator.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/StitchList.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/SQLEntityHydrator.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/SQLQuery.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/SQLQueryConstructor.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/SQLQueryEngine.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/SQLQueryEngineFactory.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/annotation/FromSubquery.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/annotation/FromTable.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/annotation/JoinTo.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metadata/SQLAnalyticView.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metadata/SQLColumn.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metadata/SQLTable.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metric/SQLMetricFunction.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metric/functions/SqlAvg.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metric/functions/SqlMax.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metric/functions/SqlMin.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metric/functions/SqlSum.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/query/SQLQueryTemplate.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/time/TimeGrain.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/EntityProjectionTranslatorTest.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/QueryValidatorTest.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/Continent.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/Country.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/CountryView.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/CountryViewNested.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/Player.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/PlayerStats.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/PlayerStatsView.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/PlayerStatsWithView.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/SubCountry.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/VideoGame.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/filter/visitor/FilterConstraintsTest.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/filter/visitor/SplitFilterExpressionVisitorTest.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/framework/AggregationDataStoreTestHarness.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/framework/SQLUnitTest.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/integration/AggregationDataStoreIntegrationTest.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/metadata/MetaDataStoreTest.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/QueryEngineTest.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/SubselectTest.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/ViewTest.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/test/resources/META-INF/persistence.xml create mode 100644 elide-datastore/elide-datastore-aggregation/src/test/resources/continent.csv create mode 100644 elide-datastore/elide-datastore-aggregation/src/test/resources/country.csv create mode 100644 elide-datastore/elide-datastore-aggregation/src/test/resources/create_tables.sql create mode 100644 elide-datastore/elide-datastore-aggregation/src/test/resources/player.csv create mode 100644 elide-datastore/elide-datastore-aggregation/src/test/resources/player_stats.csv create mode 100644 elide-datastore/elide-datastore-aggregation/src/test/resources/video_games.csv create mode 100644 elide-graphql/src/main/java/com/yahoo/elide/graphql/KeyWord.java create mode 100644 elide-graphql/src/main/java/com/yahoo/elide/graphql/parser/FragmentResolver.java create mode 100644 elide-graphql/src/main/java/com/yahoo/elide/graphql/parser/GraphQLEntityProjectionMaker.java create mode 100644 elide-graphql/src/main/java/com/yahoo/elide/graphql/parser/GraphQLProjectionInfo.java create mode 100644 elide-graphql/src/main/java/com/yahoo/elide/graphql/parser/VariableResolver.java create mode 100644 elide-graphql/src/test/resources/graphql/requests/fetch/aliasAmbiguous.graphql create mode 100644 elide-graphql/src/test/resources/graphql/requests/fetch/aliasAttribute.graphql create mode 100644 elide-graphql/src/test/resources/graphql/requests/fetch/aliasPartialQueryAmbiguous.graphql create mode 100644 elide-graphql/src/test/resources/graphql/requests/fetch/aliasRelationship.graphql create mode 100644 elide-graphql/src/test/resources/graphql/requests/fetch/aliasSameRelationship.graphql rename elide-graphql/src/test/resources/graphql/requests/fetch/{fetchWithFragment.graphql => fragmentCorrect.graphql} (100%) create mode 100644 elide-graphql/src/test/resources/graphql/requests/fetch/fragmentInline.graphql create mode 100644 elide-graphql/src/test/resources/graphql/requests/fetch/fragmentLoop.graphql create mode 100644 elide-graphql/src/test/resources/graphql/requests/fetch/fragmentUnknown.graphql create mode 100644 elide-graphql/src/test/resources/graphql/requests/fetch/rootCollectionInvalidSort.graphql create mode 100644 elide-graphql/src/test/resources/graphql/requests/fetch/rootMultiple.graphql create mode 100644 elide-graphql/src/test/resources/graphql/requests/fetch/rootUnknownField.graphql create mode 100644 elide-graphql/src/test/resources/graphql/requests/fetch/variableDefinition.graphql create mode 100644 elide-graphql/src/test/resources/graphql/requests/fetch/variableInvalidNonNull.graphql create mode 100644 elide-graphql/src/test/resources/graphql/requests/fetch/variableUnknownReference.graphql rename elide-graphql/src/test/resources/graphql/requests/replace/{replaceWithidsFails.graphql => replaceWithIdsFails.graphql} (100%) create mode 100644 elide-graphql/src/test/resources/graphql/responses/fetch/aliasAttribute.json create mode 100644 elide-graphql/src/test/resources/graphql/responses/fetch/aliasRelationship.json create mode 100644 elide-graphql/src/test/resources/graphql/responses/fetch/aliasSameRelationship.json rename elide-graphql/src/test/resources/graphql/responses/fetch/{fetchWithFragment.json => fragmentCorrect.json} (100%) create mode 100644 elide-graphql/src/test/resources/graphql/responses/fetch/variableDefinition.json delete mode 100644 elide-integration-tests/src/test/resources/EncodedErrorResponsesIT/graphQLFetchError.json create mode 100644 elide-integration-tests/src/test/resources/EncodedErrorResponsesIT/graphQLFetchErrorObjectEncoded.json create mode 100644 elide-integration-tests/src/test/resources/EncodedErrorResponsesIT/graphQLFetchErrorResponseEncoded.json create mode 100644 elide-spring/elide-spring-boot-autoconfigure/src/test/java/com/yahoo/elide/spring/models/aggregation/Stats.java rename elide-spring/elide-spring-boot-autoconfigure/src/test/java/com/yahoo/elide/spring/models/{ => jpa}/ArtifactGroup.java (95%) rename elide-spring/elide-spring-boot-autoconfigure/src/test/java/com/yahoo/elide/spring/models/{ => jpa}/ArtifactProduct.java (94%) rename elide-spring/elide-spring-boot-autoconfigure/src/test/java/com/yahoo/elide/spring/models/{ => jpa}/ArtifactVersion.java (92%) create mode 100644 elide-spring/elide-spring-boot-autoconfigure/src/test/java/com/yahoo/elide/spring/tests/AggregationStoreTest.java diff --git a/.travis.yml b/.travis.yml index 2daa1906d7..deb9b6fbfb 100644 --- a/.travis.yml +++ b/.travis.yml @@ -17,6 +17,7 @@ branches: only: - master + - elide-5.x - "/^[0-9]+\\.[0-9]+(\\.[0-9]+|-(alpha|beta)-[0-9]+)/" install: true diff --git a/checkstyle-suppressions.xml b/checkstyle-suppressions.xml index b79828e727..d69b31805c 100644 --- a/checkstyle-suppressions.xml +++ b/checkstyle-suppressions.xml @@ -10,7 +10,7 @@ "http://www.puppycrawl.com/dtds/suppressions_1_0.dtd"> - + diff --git a/elide-annotations/pom.xml b/elide-annotations/pom.xml index 802800d1a1..177db5d79a 100644 --- a/elide-annotations/pom.xml +++ b/elide-annotations/pom.xml @@ -9,7 +9,7 @@ com.yahoo.elide elide-parent-pom - 4.6.2-SNAPSHOT + 5.0.0-pr6-SNAPSHOT @@ -53,5 +53,4 @@ - diff --git a/elide-annotations/src/main/java/com/yahoo/elide/annotation/Exclude.java b/elide-annotations/src/main/java/com/yahoo/elide/annotation/Exclude.java index 043fa15b70..60dcce6e1e 100644 --- a/elide-annotations/src/main/java/com/yahoo/elide/annotation/Exclude.java +++ b/elide-annotations/src/main/java/com/yahoo/elide/annotation/Exclude.java @@ -11,7 +11,6 @@ import static java.lang.annotation.ElementType.TYPE; import static java.lang.annotation.RetentionPolicy.RUNTIME; -import java.lang.annotation.Inherited; import java.lang.annotation.Retention; import java.lang.annotation.Target; @@ -20,6 +19,5 @@ */ @Target({METHOD, FIELD, TYPE, PACKAGE}) @Retention(RUNTIME) -@Inherited public @interface Exclude { } diff --git a/elide-annotations/src/main/java/com/yahoo/elide/annotation/Include.java b/elide-annotations/src/main/java/com/yahoo/elide/annotation/Include.java index af1faadf13..4b90e31683 100644 --- a/elide-annotations/src/main/java/com/yahoo/elide/annotation/Include.java +++ b/elide-annotations/src/main/java/com/yahoo/elide/annotation/Include.java @@ -9,7 +9,6 @@ import static java.lang.annotation.ElementType.TYPE; import static java.lang.annotation.RetentionPolicy.RUNTIME; -import java.lang.annotation.Inherited; import java.lang.annotation.Retention; import java.lang.annotation.Target; @@ -18,7 +17,6 @@ */ @Target({TYPE, PACKAGE}) @Retention(RUNTIME) -@Inherited public @interface Include { /** diff --git a/elide-contrib/elide-swagger/pom.xml b/elide-contrib/elide-swagger/pom.xml index c10195708f..f845698b84 100644 --- a/elide-contrib/elide-swagger/pom.xml +++ b/elide-contrib/elide-swagger/pom.xml @@ -14,7 +14,7 @@ elide-contrib-parent-pom com.yahoo.elide - 4.6.2-SNAPSHOT + 5.0.0-pr6-SNAPSHOT @@ -42,7 +42,7 @@ com.yahoo.elide elide-core - 4.6.2-SNAPSHOT + 5.0.0-pr6-SNAPSHOT @@ -54,7 +54,7 @@ com.yahoo.elide elide-integration-tests - 4.6.2-SNAPSHOT + 5.0.0-pr6-SNAPSHOT test-jar test diff --git a/elide-contrib/elide-test-helpers/pom.xml b/elide-contrib/elide-test-helpers/pom.xml index af60193eb0..4663f250cc 100644 --- a/elide-contrib/elide-test-helpers/pom.xml +++ b/elide-contrib/elide-test-helpers/pom.xml @@ -14,7 +14,7 @@ elide-contrib-parent-pom com.yahoo.elide - 4.6.2-SNAPSHOT + 5.0.0-pr6-SNAPSHOT diff --git a/elide-contrib/elide-test-helpers/src/main/java/com/yahoo/elide/contrib/testhelpers/graphql/GraphQLDSL.java b/elide-contrib/elide-test-helpers/src/main/java/com/yahoo/elide/contrib/testhelpers/graphql/GraphQLDSL.java index 4b53d99e1d..ea101ddf7f 100644 --- a/elide-contrib/elide-test-helpers/src/main/java/com/yahoo/elide/contrib/testhelpers/graphql/GraphQLDSL.java +++ b/elide-contrib/elide-test-helpers/src/main/java/com/yahoo/elide/contrib/testhelpers/graphql/GraphQLDSL.java @@ -416,6 +416,10 @@ public static Selection field(String name, Arguments arguments, SelectionSet... return new Field(null, name, arguments, relayWrap(Arrays.asList(selectionSet))); } + public static Selection field(String alias, String name, Arguments arguments, SelectionSet... selectionSet) { + return new Field(alias, name, arguments, relayWrap(Arrays.asList(selectionSet))); + } + /** * Creates an attribute(scalar field) selection. * diff --git a/elide-contrib/elide-test-helpers/src/main/java/com/yahoo/elide/contrib/testhelpers/graphql/elements/Field.java b/elide-contrib/elide-test-helpers/src/main/java/com/yahoo/elide/contrib/testhelpers/graphql/elements/Field.java index 902d575af3..09e18b329c 100644 --- a/elide-contrib/elide-test-helpers/src/main/java/com/yahoo/elide/contrib/testhelpers/graphql/elements/Field.java +++ b/elide-contrib/elide-test-helpers/src/main/java/com/yahoo/elide/contrib/testhelpers/graphql/elements/Field.java @@ -83,7 +83,7 @@ public String toGraphQLSpec() { @Override public String toResponse() { - if (selectionSet instanceof String) { + if (selectionSet instanceof String || selectionSet instanceof Number) { // scalar response field return String.format( "\"%s\":%s", diff --git a/elide-contrib/pom.xml b/elide-contrib/pom.xml index 6343a02494..937a0d1622 100644 --- a/elide-contrib/pom.xml +++ b/elide-contrib/pom.xml @@ -14,7 +14,7 @@ elide-parent-pom com.yahoo.elide - 4.6.2-SNAPSHOT + 5.0.0-pr6-SNAPSHOT @@ -53,7 +53,7 @@ com.yahoo.elide elide-core - 4.6.2-SNAPSHOT + 5.0.0-pr6-SNAPSHOT diff --git a/elide-core/pom.xml b/elide-core/pom.xml index 406ff1292d..60b2e6f945 100644 --- a/elide-core/pom.xml +++ b/elide-core/pom.xml @@ -14,7 +14,7 @@ com.yahoo.elide elide-parent-pom - 4.6.2-SNAPSHOT + 5.0.0-pr6-SNAPSHOT @@ -183,6 +183,13 @@ test + + com.google.inject + guice + 4.2.2 + test + + ch.qos.logback logback-classic @@ -201,6 +208,7 @@ org.eclipse.jetty jetty-webapp + ${version.jetty} test diff --git a/elide-core/src/main/java/com/yahoo/elide/Elide.java b/elide-core/src/main/java/com/yahoo/elide/Elide.java index 7509d7bdad..5fa56f4451 100644 --- a/elide-core/src/main/java/com/yahoo/elide/Elide.java +++ b/elide-core/src/main/java/com/yahoo/elide/Elide.java @@ -23,6 +23,7 @@ import com.yahoo.elide.core.exceptions.UnableToAddSerdeException; import com.yahoo.elide.extensions.JsonApiPatch; import com.yahoo.elide.extensions.PatchRequestScope; +import com.yahoo.elide.jsonapi.EntityProjectionMaker; import com.yahoo.elide.jsonapi.JsonApiMapper; import com.yahoo.elide.jsonapi.models.JsonApiDocument; import com.yahoo.elide.parsers.BaseVisitor; @@ -165,6 +166,8 @@ public ElideResponse get(String path, MultivaluedMap queryParams return handleRequest(true, opaqueUser, dataStore::beginReadTransaction, (tx, user) -> { JsonApiDocument jsonApiDoc = new JsonApiDocument(); RequestScope requestScope = new RequestScope(path, jsonApiDoc, tx, user, queryParams, elideSettings); + requestScope.setEntityProjection(new EntityProjectionMaker(elideSettings.getDictionary(), + requestScope).parsePath(path)); BaseVisitor visitor = new GetVisitor(requestScope); return visit(path, requestScope, visitor); }); @@ -183,6 +186,8 @@ public ElideResponse post(String path, String jsonApiDocument, Object opaqueUser return handleRequest(false, opaqueUser, dataStore::beginTransaction, (tx, user) -> { JsonApiDocument jsonApiDoc = mapper.readJsonApiDocument(jsonApiDocument); RequestScope requestScope = new RequestScope(path, jsonApiDoc, tx, user, null, elideSettings); + requestScope.setEntityProjection(new EntityProjectionMaker(elideSettings.getDictionary(), + requestScope).parsePath(path)); BaseVisitor visitor = new PostVisitor(requestScope); return visit(path, requestScope, visitor); }); @@ -217,6 +222,8 @@ public ElideResponse patch(String contentType, String accept, handler = (tx, user) -> { JsonApiDocument jsonApiDoc = mapper.readJsonApiDocument(jsonApiDocument); RequestScope requestScope = new RequestScope(path, jsonApiDoc, tx, user, null, elideSettings); + requestScope.setEntityProjection(new EntityProjectionMaker(elideSettings.getDictionary(), + requestScope).parsePath(path)); BaseVisitor visitor = new PatchVisitor(requestScope); return visit(path, requestScope, visitor); }; @@ -239,6 +246,8 @@ public ElideResponse delete(String path, String jsonApiDocument, Object opaqueUs ? new JsonApiDocument() : mapper.readJsonApiDocument(jsonApiDocument); RequestScope requestScope = new RequestScope(path, jsonApiDoc, tx, user, null, elideSettings); + requestScope.setEntityProjection(new EntityProjectionMaker(elideSettings.getDictionary(), + requestScope).parsePath(path)); BaseVisitor visitor = new DeleteVisitor(requestScope); return visit(path, requestScope, visitor); }); diff --git a/elide-core/src/main/java/com/yahoo/elide/core/ArgumentType.java b/elide-core/src/main/java/com/yahoo/elide/core/ArgumentType.java new file mode 100644 index 0000000000..ffe0773280 --- /dev/null +++ b/elide-core/src/main/java/com/yahoo/elide/core/ArgumentType.java @@ -0,0 +1,23 @@ +/* + * Copyright 2015, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.core; + +import lombok.Getter; + +/** + * Argument Type wraps an argument to the type of value it accepts. + */ +public class ArgumentType { + @Getter + private String name; + @Getter + private Class type; + + public ArgumentType(String name, Class type) { + this.name = name; + this.type = type; + } +} diff --git a/elide-core/src/main/java/com/yahoo/elide/core/CheckInstantiator.java b/elide-core/src/main/java/com/yahoo/elide/core/CheckInstantiator.java index 43773fe074..c1b25701ea 100644 --- a/elide-core/src/main/java/com/yahoo/elide/core/CheckInstantiator.java +++ b/elide-core/src/main/java/com/yahoo/elide/core/CheckInstantiator.java @@ -6,6 +6,7 @@ package com.yahoo.elide.core; +import com.yahoo.elide.Injector; import com.yahoo.elide.security.checks.Check; import java.util.Objects; @@ -25,7 +26,7 @@ public interface CheckInstantiator { */ default Check getCheck(EntityDictionary dictionary, String checkName) { Class checkCls = dictionary.getCheck(checkName); - return instantiateCheck(checkCls); + return instantiateCheck(checkCls, dictionary.getInjector()); } /** @@ -34,9 +35,11 @@ default Check getCheck(EntityDictionary dictionary, String checkName) { * @return the instance of the check * @throws IllegalArgumentException if the check class cannot be instantiated with a zero argument constructor */ - default Check instantiateCheck(Class checkCls) { + default Check instantiateCheck(Class checkCls, Injector injector) { try { - return Objects.requireNonNull(checkCls).newInstance(); + Check check = Objects.requireNonNull(checkCls).newInstance(); + injector.inject(check); + return check; } catch (InstantiationException | IllegalAccessException | NullPointerException e) { String checkName = (checkCls != null) ? checkCls.getName() : "null"; throw new IllegalArgumentException("Could not instantiate specified check '" + checkName + "'.", e); diff --git a/elide-core/src/main/java/com/yahoo/elide/core/DataStoreTransaction.java b/elide-core/src/main/java/com/yahoo/elide/core/DataStoreTransaction.java index 9286603661..8344da8fc8 100644 --- a/elide-core/src/main/java/com/yahoo/elide/core/DataStoreTransaction.java +++ b/elide-core/src/main/java/com/yahoo/elide/core/DataStoreTransaction.java @@ -8,14 +8,15 @@ import com.yahoo.elide.core.filter.InPredicate; import com.yahoo.elide.core.filter.expression.AndFilterExpression; import com.yahoo.elide.core.filter.expression.FilterExpression; -import com.yahoo.elide.core.pagination.Pagination; import com.yahoo.elide.core.sort.Sorting; +import com.yahoo.elide.request.Attribute; +import com.yahoo.elide.request.EntityProjection; +import com.yahoo.elide.request.Relationship; import com.yahoo.elide.security.User; import java.io.Closeable; import java.io.Serializable; import java.util.Iterator; -import java.util.Optional; import java.util.Set; /** @@ -112,19 +113,21 @@ default T createNewObject(Class entityClass) { } /** - * Loads an object by ID. + * Loads an object by ID. The reason we support both load by ID and load by filter is that + * some legacy stores are optimized to load by ID. * - * @param entityClass the type of class to load + * @param entityProjection the collection to load. * @param id - the ID of the object to load. - * @param filterExpression - security filters that can be evaluated in the data store. * @param scope - the current request scope * It is optional for the data store to attempt evaluation. * @return the loaded object if it exists AND any provided security filters pass. */ - default Object loadObject(Class entityClass, - Serializable id, - Optional filterExpression, - RequestScope scope) { + default Object loadObject(EntityProjection entityProjection, + Serializable id, + RequestScope scope) { + Class entityClass = entityProjection.getType(); + FilterExpression filterExpression = entityProjection.getFilterExpression(); + EntityDictionary dictionary = scope.getDictionary(); Class idType = dictionary.getIdType(entityClass); String idField = dictionary.getIdFieldName(entityClass); @@ -132,14 +135,15 @@ default Object loadObject(Class entityClass, new Path.PathElement(entityClass, idType, idField), id ); - FilterExpression joinedFilterExpression = filterExpression - .map(fe -> (FilterExpression) new AndFilterExpression(idFilter, fe)) - .orElse(idFilter); - Iterable results = loadObjects(entityClass, - Optional.of(joinedFilterExpression), - Optional.empty(), - Optional.empty(), + FilterExpression joinedFilterExpression = (filterExpression != null) + ? new AndFilterExpression(idFilter, filterExpression) + : idFilter; + + Iterable results = loadObjects(entityProjection.copyOf() + .filterExpression(joinedFilterExpression) + .build(), scope); + Iterator it = results == null ? null : results.iterator(); if (it != null && it.hasNext()) { return it.next(); @@ -150,19 +154,12 @@ default Object loadObject(Class entityClass, /** * Loads a collection of objects. * - * @param entityClass - the class to load - * @param filterExpression - filters that can be evaluated in the data store. - * It is optional for the data store to attempt evaluation. - * @param sorting - sorting which can be pushed down to the data store. - * @param pagination - pagination which can be pushed down to the data store. + * @param entityProjection - the class to load * @param scope - contains request level metadata. * @return a collection of the loaded objects */ Iterable loadObjects( - Class entityClass, - Optional filterExpression, - Optional sorting, - Optional pagination, + EntityProjection entityProjection, RequestScope scope); /** @@ -170,25 +167,18 @@ Iterable loadObjects( * * @param relationTx - The datastore that governs objects of the relationhip's type. * @param entity - The object which owns the relationship. - * @param relationName - name of the relationship. - * @param filterExpression - filtering which can be pushed down to the data store. - * It is optional for the data store to attempt evaluation. - * @param sorting - sorting which can be pushed down to the data store. - * @param pagination - pagination which can be pushed down to the data store. + * @param relationship - the relationship to fetch. * @param scope - contains request level metadata. * @return the object in the relation */ default Object getRelation( DataStoreTransaction relationTx, Object entity, - String relationName, - Optional filterExpression, - Optional sorting, - Optional pagination, + Relationship relationship, RequestScope scope) { - return PersistentResource.getValue(entity, relationName, scope); - } + return PersistentResource.getValue(entity, relationship.getName(), scope); + } /** * Elide core will update the in memory representation of the objects to the requested state. @@ -230,14 +220,14 @@ default void updateToOneRelation(DataStoreTransaction relationTx, * Get an attribute from an object. * * @param entity - The object which owns the attribute. - * @param attributeName - name of the attribute. + * @param attribute - The attribute to fetch * @param scope - contains request level metadata. * @return the value of the attribute */ default Object getAttribute(Object entity, - String attributeName, + Attribute attribute, RequestScope scope) { - return PersistentResource.getValue(entity, attributeName, scope); + return PersistentResource.getValue(entity, attribute.getName(), scope); } @@ -248,13 +238,11 @@ default Object getAttribute(Object entity, * This function allow a data store to optionally persist the attribute if needed. * * @param entity - The object which owns the attribute. - * @param attributeName - name of the attribute. - * @param attributeValue - the desired attribute value. + * @param attribute - the attribute to set. * @param scope - contains request level metadata. */ default void setAttribute(Object entity, - String attributeName, - Object attributeValue, + Attribute attribute, RequestScope scope) { } @@ -270,7 +258,7 @@ default FeatureSupport supportsFiltering(Class entityClass, FilterExpression /** * Whether or not the transaction can sort the provided class. - * @param entityClass + * @param entityClass The entity class that is being sorted. * @return true if sorting is possible */ default boolean supportsSorting(Class entityClass, Sorting sorting) { @@ -279,7 +267,7 @@ default boolean supportsSorting(Class entityClass, Sorting sorting) { /** * Whether or not the transaction can paginate the provided class. - * @param entityClass + * @param entityClass The entity class that is being paged. * @return true if pagination is possible */ default boolean supportsPagination(Class entityClass) { diff --git a/elide-core/src/main/java/com/yahoo/elide/core/EntityBinding.java b/elide-core/src/main/java/com/yahoo/elide/core/EntityBinding.java index ab35dac888..e9221bb753 100644 --- a/elide-core/src/main/java/com/yahoo/elide/core/EntityBinding.java +++ b/elide-core/src/main/java/com/yahoo/elide/core/EntityBinding.java @@ -36,7 +36,6 @@ import org.apache.commons.lang3.tuple.Pair; import lombok.Getter; -import lombok.Setter; import java.lang.annotation.Annotation; import java.lang.reflect.AccessibleObject; @@ -51,14 +50,15 @@ import java.util.Collection; import java.util.Collections; import java.util.Deque; +import java.util.HashSet; import java.util.List; import java.util.Optional; +import java.util.Set; import java.util.concurrent.ConcurrentHashMap; import java.util.concurrent.ConcurrentLinkedDeque; import java.util.function.Predicate; import java.util.stream.Collectors; import java.util.stream.Stream; - import javax.persistence.AccessType; import javax.persistence.CascadeType; import javax.persistence.Column; @@ -95,9 +95,6 @@ public class EntityBinding { @Getter private Class idType; @Getter - @Setter - private Initializer initializer; - @Getter private AccessType accessType; public final EntityPermissions entityPermissions; @@ -116,10 +113,12 @@ public class EntityBinding { public final ConcurrentHashMap> fieldsToTypes = new ConcurrentHashMap<>(); public final ConcurrentHashMap aliasesToFields = new ConcurrentHashMap<>(); public final ConcurrentHashMap requestScopeableMethods = new ConcurrentHashMap<>(); + public final ConcurrentHashMap> attributeArguments = new ConcurrentHashMap<>(); public final ConcurrentHashMap annotations = new ConcurrentHashMap<>(); public static final EntityBinding EMPTY_BINDING = new EntityBinding(); + public static final Set EMPTY_ATTRIBUTES_ARGS = Collections.unmodifiableSet(new HashSet<>()); private static final String ALL_FIELDS = "*"; /* empty binding constructor */ @@ -601,4 +600,35 @@ private List> getInheritedTypes(Class entityClass) { return results; } + + + /** + * Add a collection of arguments to the attributes of this Entity. + * @param attribute attribute name to which argument has to be added + * @param arguments Set of Argument Type for the attribute + */ + public void addArgumentsToAttribute(String attribute, Set arguments) { + AccessibleObject fieldObject = fieldsToValues.get(attribute); + if (fieldObject != null && arguments != null) { + Set existingArgs = attributeArguments.get(fieldObject); + if (existingArgs != null) { + //Replace any argument names with new value + existingArgs.addAll(arguments); + } else { + attributeArguments.put(fieldObject, new HashSet<>(arguments)); + } + } + } + + /** + * Returns the Collection of all attributes of an argument. + * @param attribute Name of the argument for ehich arguments are to be retrieved. + * @return A Set of ArgumentType for the given attribute. + */ + public Set getAttributeArguments(String attribute) { + AccessibleObject fieldObject = fieldsToValues.get(attribute); + return (fieldObject != null) + ? attributeArguments.getOrDefault(fieldObject, EMPTY_ATTRIBUTES_ARGS) + : EMPTY_ATTRIBUTES_ARGS; + } } diff --git a/elide-core/src/main/java/com/yahoo/elide/core/EntityDictionary.java b/elide-core/src/main/java/com/yahoo/elide/core/EntityDictionary.java index bca8faf241..9f552250ec 100644 --- a/elide-core/src/main/java/com/yahoo/elide/core/EntityDictionary.java +++ b/elide-core/src/main/java/com/yahoo/elide/core/EntityDictionary.java @@ -20,6 +20,7 @@ import com.yahoo.elide.core.exceptions.InternalServerErrorException; import com.yahoo.elide.core.exceptions.InvalidAttributeException; import com.yahoo.elide.functions.LifeCycleHook; +import com.yahoo.elide.security.FilterExpressionCheck; import com.yahoo.elide.security.checks.Check; import com.yahoo.elide.security.checks.prefab.Collections.AppendOnly; import com.yahoo.elide.security.checks.prefab.Collections.RemoveOnly; @@ -31,10 +32,12 @@ import com.google.common.collect.BiMap; import com.google.common.collect.HashBiMap; import com.google.common.collect.Maps; +import com.google.common.collect.Sets; import org.antlr.v4.runtime.tree.ParseTree; import org.apache.commons.lang3.StringUtils; +import lombok.Getter; import lombok.extern.slf4j.Slf4j; import java.lang.annotation.Annotation; @@ -63,7 +66,9 @@ import javax.persistence.AccessType; import javax.persistence.CascadeType; +import javax.persistence.Column; import javax.persistence.Entity; +import javax.persistence.JoinColumn; import javax.persistence.Transient; import javax.ws.rs.WebApplicationException; @@ -81,6 +86,8 @@ public class EntityDictionary { protected final CopyOnWriteArrayList> bindEntityRoots = new CopyOnWriteArrayList<>(); protected final ConcurrentHashMap, List>> subclassingEntities = new ConcurrentHashMap<>(); protected final BiMap> checkNames; + + @Getter protected final Injector injector; public final static String REGULAR_ID_NAME = "id"; @@ -95,7 +102,24 @@ public class EntityDictionary { * to their implementing classes */ public EntityDictionary(Map> checks) { - this(checks, null); + this.checkNames = Maps.synchronizedBiMap(HashBiMap.create(checks)); + initializeChecks(); + + //Default injector only injects Elide internals. + this.injector = new Injector() { + @Override + public void inject(Object entity) { + if (entity instanceof FilterExpressionCheck) { + try { + Field field = FilterExpressionCheck.class.getDeclaredField("dictionary"); + field.setAccessible(true); + field.set(entity, EntityDictionary.this); + } catch (NoSuchFieldException | IllegalAccessException e) { + throw new IllegalStateException(e); + } + } + } + }; } /** @@ -109,17 +133,20 @@ public EntityDictionary(Map> checks) { * initialize Elide models. */ public EntityDictionary(Map> checks, Injector injector) { - checkNames = Maps.synchronizedBiMap(HashBiMap.create(checks)); + this.checkNames = Maps.synchronizedBiMap(HashBiMap.create(checks)); + initializeChecks(); + this.injector = injector; + } + private void initializeChecks() { addPrefabCheck("Prefab.Role.All", Role.ALL.class); addPrefabCheck("Prefab.Role.None", Role.NONE.class); addPrefabCheck("Prefab.Collections.AppendOnly", AppendOnly.class); addPrefabCheck("Prefab.Collections.RemoveOnly", RemoveOnly.class); addPrefabCheck("Prefab.Common.UpdateOnCreate", Common.UpdateOnCreate.class); - - this.injector = injector; } + private void addPrefabCheck(String alias, Class checkClass) { if (checkNames.containsKey(alias) || checkNames.inverse().containsKey(checkClass)) { return; @@ -267,8 +294,8 @@ public ParseTree getPermissionsForClass(Class resourceClass, Class resourceClass, - String field, - Class annotationClass) { + String field, + Class annotationClass) { EntityBinding binding = getEntityBinding(resourceClass); return binding.entityPermissions.getFieldChecksForPermission(field, annotationClass); } @@ -779,28 +806,10 @@ public String getNameFromAlias(Object entity, String alias) { */ public void initializeEntity(T entity) { if (entity != null) { - @SuppressWarnings("unchecked") - Initializer initializer = getEntityBinding(entity.getClass()).getInitializer(); - if (initializer != null) { - initializer.initialize(entity); - } else if (injector != null) { - injector.inject(entity); - } + injector.inject(entity); } } - /** - * Bind a particular initializer to a class. - * - * @param the type parameter - * @param initializer Initializer to use for class - * @param cls Class to bind initialization - */ - public void bindInitializer(Initializer initializer, Class cls) { - bindIfUnbound(cls); - getEntityBinding(cls).setInitializer(initializer); - } - /** * Returns whether or not an entity is shareable. * @@ -1040,7 +1049,7 @@ public Collection getIdAnnotations(Object value) { } /** - * Follow for this class or super-class for Entity annotation. + * Follow for this class or super-class for JPA {@link Entity} annotation. * * @param objClass provided class * @return class with Entity annotation @@ -1062,6 +1071,12 @@ public Class lookupEntityClass(Class objClass) { public Class lookupIncludeClass(Class objClass) { Annotation first = getFirstAnnotation(objClass, Arrays.asList(Exclude.class, Include.class)); if (first instanceof Include) { + Class declaringClass = lookupAnnotationDeclarationClass(objClass, Include.class); + if (declaringClass != null) { + return declaringClass; + } + + //If we didn't find Include declared on a class, it must be declared at the package level. return objClass; } return null; @@ -1082,6 +1097,7 @@ public Class lookupAnnotationDeclarationClass(Class objClass, Class objClass) { } + /** + * Check whether a class is a JPA entity + * + * @param objClass class + * @return True if it is a JPA entity + */ + public final boolean isJPAEntity(Class objClass) { + try { + lookupEntityClass(objClass); + return true; + } catch (IllegalArgumentException e) { + return false; + } + } + /** * Retrieve the accessible object for a field from a target object. * @@ -1313,7 +1344,7 @@ public List walkEntityGraph(Set> entities, Function, T /** * Returns whether or not a class is already bound. - * @param cls + * @param cls The class to verify. * @return true if the class is bound. False otherwise. */ public boolean hasBinding(Class cls) { @@ -1464,6 +1495,36 @@ private Map coerceMap(Object target, Map values, String fieldName) { return result; } + /** + * Returns whether or not a specified annotation is present on an entity field or its corresponding method. + * + * @param fieldName The entity field + * @param annotationClass The provided annotation class + * + * @param The type of the {@code annotationClass} + * + * @return {@code true} if the field is annotated by the {@code annotationClass} + */ + public boolean attributeOrRelationAnnotationExists( + Class cls, + String fieldName, + Class annotationClass + ) { + return getAttributeOrRelationAnnotation(cls, annotationClass, fieldName) != null; + } + + /** + * Returns whether or not a specified field exists in an entity. + * + * @param cls The entity + * @param fieldName The provided field to check + * + * @return {@code true} if the field exists in the entity + */ + public boolean isValidField(Class cls, String fieldName) { + return getAllFields(cls).contains(fieldName); + } + private boolean isValidParameterizedMap(Map values, Class keyType, Class valueType) { for (Map.Entry entry : values.entrySet()) { Object key = entry.getKey(); @@ -1481,8 +1542,64 @@ private boolean isValidParameterizedMap(Map values, Class keyType, Clas * @param entityClass the class to bind. */ private void bindIfUnbound(Class entityClass) { + /* This is safe to call with non-proxy objects. Not safe to call with ORM proxy objects. */ + if (! isClassBound(entityClass)) { bindEntity(entityClass); } } + + /** + * Add a collection of argument to the attributes + * @param cls The entity + * @param attributeName attribute name to which argument has to be added + * @param arguments Set of Argument type containing name and type of each argument. + */ + public void addArgumentsToAttribute(Class cls, String attributeName, Set arguments) { + getEntityBinding(cls).addArgumentsToAttribute(attributeName, arguments); + } + + /** + * Add a single argument to the attribute + * @param cls The entity + * @param attributeName attribute name to which argument has to be added + * @param argument A single argument + */ + public void addArgumentToAttribute(Class cls, String attributeName, ArgumentType argument) { + this.addArgumentsToAttribute(cls, attributeName, Sets.newHashSet(argument)); + } + + /** + * Returns the Collection of all attributes of an argument. + * @param cls The entity + * @param attributeName Name of the argument for ehich arguments are to be retrieved. + * @return A Set of ArgumentType for the given attribute. + */ + public Set getAttributeArguments(Class cls, String attributeName) { + return entityBindings.getOrDefault(cls, EMPTY_BINDING).getAttributeArguments(attributeName); + } + + /** + * Get column name using JPA. + * + * @param cls The entity class. + * @param fieldName The entity attribute. + * @return The jpa column name. + */ + public String getAnnotatedColumnName(Class cls, String fieldName) { + Column[] column = getAttributeOrRelationAnnotations(cls, Column.class, fieldName); + + // this would only be valid for dimension columns + JoinColumn[] joinColumn = getAttributeOrRelationAnnotations(cls, JoinColumn.class, fieldName); + + if (column == null || column.length == 0) { + if (joinColumn == null || joinColumn.length == 0) { + return fieldName; + } else { + return joinColumn[0].name(); + } + } else { + return column[0].name(); + } + } } diff --git a/elide-core/src/main/java/com/yahoo/elide/core/Initializer.java b/elide-core/src/main/java/com/yahoo/elide/core/Initializer.java deleted file mode 100644 index 79aa554139..0000000000 --- a/elide-core/src/main/java/com/yahoo/elide/core/Initializer.java +++ /dev/null @@ -1,22 +0,0 @@ -/* - * Copyright 2017, Yahoo Inc. - * Licensed under the Apache License, Version 2.0 - * See LICENSE file in project root for terms. - */ -package com.yahoo.elide.core; - -/** - * Used to perform any additional initialization required on entity beans which is not - * possible at time of construction. - * @param bean type - */ -@FunctionalInterface -public interface Initializer { - - /** - * Initialize an entity bean. - * - * @param entity Entity bean to initialize - */ - void initialize(T entity); -} diff --git a/elide-core/src/main/java/com/yahoo/elide/core/PersistentResource.java b/elide-core/src/main/java/com/yahoo/elide/core/PersistentResource.java index a007e66e6b..87d4d70ee4 100644 --- a/elide-core/src/main/java/com/yahoo/elide/core/PersistentResource.java +++ b/elide-core/src/main/java/com/yahoo/elide/core/PersistentResource.java @@ -31,6 +31,9 @@ import com.yahoo.elide.jsonapi.models.ResourceIdentifier; import com.yahoo.elide.jsonapi.models.SingleElementSet; import com.yahoo.elide.parsers.expression.CanPaginateVisitor; +import com.yahoo.elide.request.Argument; +import com.yahoo.elide.request.Attribute; +import com.yahoo.elide.request.EntityProjection; import com.yahoo.elide.security.ChangeSpec; import com.yahoo.elide.security.permissions.ExpressionResult; import com.yahoo.elide.utils.coerce.CoerceUtil; @@ -38,13 +41,11 @@ import com.fasterxml.jackson.annotation.JsonIgnore; import com.google.common.base.Preconditions; import com.google.common.collect.Sets; - import org.apache.commons.collections4.CollectionUtils; import org.apache.commons.collections4.IterableUtils; import org.apache.commons.lang3.StringUtils; import lombok.NonNull; - import java.io.Serializable; import java.lang.annotation.Annotation; import java.util.ArrayList; @@ -77,6 +78,7 @@ public class PersistentResource implements com.yahoo.elide.security.Persisten private final DataStoreTransaction transaction; private final RequestScope requestScope; private int hashCode = 0; + static final String CLASS_NO_FIELD = ""; /** @@ -95,6 +97,21 @@ public String toString() { return String.format("PersistentResource{type=%s, id=%s}", type, uuid.orElse(getId())); } + /** + * Create a resource in the database. + * @param entityClass the entity class + * @param requestScope the request scope + * @param uuid the (optional) uuid + * @param object type + * @return persistent resource + */ + public static PersistentResource createObject( + Class entityClass, + RequestScope requestScope, + Optional uuid) { + return createObject(null, entityClass, requestScope, uuid); + } + /** * Create a resource in the database. * @param parent - The immediate ancestor in the lineage or null if this is a root. @@ -110,7 +127,7 @@ public static PersistentResource createObject( RequestScope requestScope, Optional uuid) { - //instead of calling transcation.createObject, create the new object here. + //instead of calling transaction.createObject, create the new object here. T obj = requestScope.getTransaction().createNewObject(entityClass); String id = uuid.orElse(null); @@ -150,7 +167,12 @@ public static PersistentResource createObject( * @param id the id * @param scope the request scope */ - public PersistentResource(@NonNull T obj, PersistentResource parent, String id, @NonNull RequestScope scope) { + public PersistentResource( + @NonNull T obj, + PersistentResource parent, + String id, + @NonNull RequestScope scope + ) { this.obj = obj; this.uuid = Optional.ofNullable(id); this.lineage = parent != null ? new ResourceLineage(parent.lineage, parent) : new ResourceLineage(); @@ -161,7 +183,7 @@ public PersistentResource(@NonNull T obj, PersistentResource parent, String id, dictionary.initializeEntity(obj); } - /** + /** * Check whether an id matches for this persistent resource. * * @param checkId the check id @@ -183,7 +205,7 @@ public boolean matchesId(String checkId) { /** * Load an single entity from the DB. * - * @param loadClass resource type + * @param projection What to load from the DB. * @param id the id * @param requestScope the request scope * @param type of resource @@ -192,14 +214,17 @@ public boolean matchesId(String checkId) { */ @SuppressWarnings("resource") @NonNull public static PersistentResource loadRecord( - Class loadClass, String id, RequestScope requestScope) - throws InvalidObjectIdentifierException { - Preconditions.checkNotNull(loadClass); + EntityProjection projection, + String id, + RequestScope requestScope + ) throws InvalidObjectIdentifierException { + Preconditions.checkNotNull(projection); Preconditions.checkNotNull(id); Preconditions.checkNotNull(requestScope); DataStoreTransaction tx = requestScope.getTransaction(); EntityDictionary dictionary = requestScope.getDictionary(); + Class loadClass = projection.getType(); // Check the resource cache if exists Object obj = requestScope.getObjectById(dictionary.getJsonAliasFor(loadClass), id); @@ -208,15 +233,24 @@ public boolean matchesId(String checkId) { Optional permissionFilter = getPermissionFilterExpression(loadClass, requestScope); Class idType = dictionary.getIdType(loadClass); - obj = tx.loadObject(loadClass, (Serializable) CoerceUtil.coerce(id, idType), - permissionFilter, requestScope); + + projection = projection + .copyOf() + .filterExpression(permissionFilter.orElse(null)) + .build(); + + obj = tx.loadObject(projection, (Serializable) CoerceUtil.coerce(id, idType), requestScope); if (obj == null) { throw new InvalidObjectIdentifierException(id, dictionary.getJsonAliasFor(loadClass)); } } PersistentResource resource = new PersistentResource<>( - loadClass.cast(obj), null, requestScope.getUUIDFor(obj), requestScope); + (T) obj, + null, + requestScope.getUUIDFor(obj), + requestScope); + // No need to have read access for a newly created object if (!requestScope.getNewResources().contains(resource)) { resource.checkFieldAwarePermissions(ReadPermission.class); @@ -245,25 +279,26 @@ private static Optional getPermissionFilterExpression(Clas /** * Load a collection from the datastore. * - * @param loadClass the load class + * @param projection the projection to load * @param requestScope the request scope * @param ids a list of object identifiers to optionally load. Can be empty. * @return a filtered collection of resources loaded from the datastore. */ public static Set loadRecords( - Class loadClass, + EntityProjection projection, List ids, - Optional filter, - Optional sorting, - Optional pagination, RequestScope requestScope) { + Class loadClass = projection.getType(); + Pagination pagination = projection.getPagination(); + Sorting sorting = projection.getSorting(); + + FilterExpression filterExpression = projection.getFilterExpression(); + EntityDictionary dictionary = requestScope.getDictionary(); - FilterExpression filterExpression; DataStoreTransaction tx = requestScope.getTransaction(); - if (shouldSkipCollection(loadClass, ReadPermission.class, requestScope)) { if (ids.isEmpty()) { return Collections.emptySet(); @@ -271,8 +306,7 @@ public static Set loadRecords( throw new InvalidObjectIdentifierException(ids.toString(), dictionary.getJsonAliasFor(loadClass)); } - - if (pagination.isPresent() && !pagination.get().isDefaultInstance() + if (pagination != null && !pagination.isDefaultInstance() && !CanPaginateVisitor.canPaginate(loadClass, dictionary, requestScope)) { throw new InvalidPredicateException(String.format("Cannot paginate %s", dictionary.getJsonAliasFor(loadClass))); @@ -289,11 +323,9 @@ public static Set loadRecords( FilterExpression idExpression = buildIdFilterExpression(ids, loadClass, dictionary, requestScope); // Combine filters if necessary - filterExpression = filter + filterExpression = Optional.ofNullable(filterExpression) .map(fe -> (FilterExpression) new AndFilterExpression(idExpression, fe)) .orElse(idExpression); - } else { - filterExpression = filter.orElse(null); } Optional permissionFilter = getPermissionFilterExpression(loadClass, requestScope); @@ -305,9 +337,15 @@ public static Set loadRecords( } } - Set existingResources = filter(ReadPermission.class, filter, - new PersistentResourceSet(tx.loadObjects(loadClass, Optional.ofNullable(filterExpression), sorting, - pagination.map(p -> p.evaluate(loadClass)), requestScope), requestScope)); + EntityProjection modifiedProjection = projection + .copyOf() + .filterExpression(filterExpression) + .sorting(sorting) + .pagination(Optional.ofNullable(pagination).map(p -> p.evaluate(loadClass)).orElse(null)) + .build(); + + Set existingResources = filter(ReadPermission.class, + new PersistentResourceSet(tx.loadObjects(modifiedProjection, requestScope), requestScope)); Set allResources = Sets.union(newResources, existingResources); @@ -340,7 +378,13 @@ public boolean updateAttribute(String fieldName, Object newVal) { this.markDirty(); //Hooks for customize logic for setAttribute/Relation if (dictionary.isAttribute(obj.getClass(), fieldName)) { - transaction.setAttribute(obj, fieldName, newVal, requestScope); + transaction.setAttribute(obj, Attribute.builder() + .name(fieldName) + .type(fieldClass) + .argument(Argument.builder() + .name("_UNUSED_") + .value(newVal).build()) + .build(), requestScope); } return true; } @@ -789,49 +833,44 @@ public Optional getUUID() { * * NOTE: Filter expressions for this type are _not_ applied at this level. * - * @param relation relation name + * @param relationship The relationship * @param id single id to lookup * @return The PersistentResource of the sought id or null if does not exist. */ - public PersistentResource getRelation(String relation, String id) { - Set resources = getRelation(relation, Collections.singletonList(id), - Optional.empty(), Optional.empty(), Optional.empty()); - if (resources.isEmpty()) { + public PersistentResource getRelation(com.yahoo.elide.request.Relationship relationship, String id) { + Set resources = getRelation(Collections.singletonList(id), relationship); + + if (resources.isEmpty()) { return null; - } - // If this is an in-memory object (i.e. UUID being created within tx), datastore may not be able to filter. - // If we get multiple results back, make sure we find the right id first. - for (PersistentResource resource : resources) { - if (resource.matchesId(id)) { - return resource; - } - } - return null; + } + // If this is an in-memory object (i.e. UUID being created within tx), datastore may not be able to filter. + // If we get multiple results back, make sure we find the right id first. + for (PersistentResource resource : resources) { + if (resource.matchesId(id)) { + return resource; + } + } + return null; } /** - * Load a single entity relation from the PersistentResource. Example: GET /book/2 + * Load a relation from the PersistentResource. * - * @param relation the relation + * @param relationship the relation * @param ids a list of object identifiers to optionally load. Can be empty. * @return PersistentResource relation */ - public Set getRelation(String relation, - List ids, - Optional filter, - Optional sorting, - Optional pagination) { + public Set getRelation(List ids, com.yahoo.elide.request.Relationship relationship) { - FilterExpression filterExpression; + FilterExpression filterExpression = Optional.ofNullable(relationship.getProjection().getFilterExpression()) + .orElse(null); - Class entityType = dictionary.getParameterizedType(getResourceClass(), relation); + assertRelationshipExists(relationship.getName()); + Class entityType = dictionary.getParameterizedType(getResourceClass(), relationship.getName()); Set newResources = new LinkedHashSet<>(); /* If this is a bulk edit request and the ID we are fetching for is newly created... */ - if (entityType == null) { - throw new InvalidAttributeException(relation, type); - } if (!ids.isEmpty()) { // Fetch our set of new resources that we know about since we can't find them in the datastore newResources = requestScope.getNewPersistentResources().stream() @@ -842,18 +881,21 @@ public Set getRelation(String relation, FilterExpression idExpression = buildIdFilterExpression(ids, entityType, dictionary, requestScope); // Combine filters if necessary - filterExpression = filter + filterExpression = Optional.ofNullable(relationship.getProjection().getFilterExpression()) .map(fe -> (FilterExpression) new AndFilterExpression(idExpression, fe)) .orElse(idExpression); - } else { - filterExpression = filter.orElse(null); } // TODO: Filter on new resources? // TODO: Update pagination to subtract the number of new resources created? - Set existingResources = filter(ReadPermission.class, filter, - getRelation(relation, Optional.ofNullable(filterExpression), sorting, pagination, true)); + Set existingResources = filter(ReadPermission.class, + + getRelation(relationship.copyOf() + .projection(relationship.getProjection().copyOf() + .filterExpression(filterExpression) + .build()) + .build(), true)); // TODO: Sort again in memory now that two sets are glommed together? @@ -864,7 +906,7 @@ public Set getRelation(String relation, Set missedIds = Sets.difference(new HashSet<>(ids), allExpectedIds); if (!missedIds.isEmpty()) { - throw new InvalidObjectIdentifierException(missedIds.toString(), relation); + throw new InvalidObjectIdentifierException(missedIds.toString(), relationship.getName()); } return allResources; @@ -904,60 +946,73 @@ private static FilterExpression buildIdFilterExpression(List ids, /** * Get collection of resources from relation field. * - * @param relationName field + * @param relationship relationship * @return collection relation */ - public Set getRelationCheckedFiltered(String relationName, - Optional filterExpression, - Optional sorting, - Optional pagination) { - - return filter(ReadPermission.class, filterExpression, - getRelation(relationName, filterExpression, sorting, pagination, true)); + public Set getRelationCheckedFiltered(com.yahoo.elide.request.Relationship relationship) { + return filter(ReadPermission.class, + getRelation(relationship, true)); } private Set getRelationUncheckedUnfiltered(String relationName) { - return getRelation(relationName, Optional.empty(), Optional.empty(), Optional.empty(), false); + assertRelationshipExists(relationName); + return getRelation(com.yahoo.elide.request.Relationship.builder() + .name(relationName) + .alias(relationName) + .projection(EntityProjection.builder() + .type(dictionary.getParameterizedType(getResourceClass(), relationName)) + .build()) + .build(), false); } private Set getRelationCheckedUnfiltered(String relationName) { - return getRelation(relationName, Optional.empty(), Optional.empty(), Optional.empty(), true); + assertRelationshipExists(relationName); + return getRelation(com.yahoo.elide.request.Relationship.builder() + .name(relationName) + .alias(relationName) + .projection(EntityProjection.builder() + .type(dictionary.getParameterizedType(getResourceClass(), relationName)) + .build()) + .build(), true); + } + + private void assertRelationshipExists(String relationName) { + if (relationName == null || dictionary.getParameterizedType(obj, relationName) == null) { + throw new InvalidAttributeException(relationName, this.getType()); + } } - private Set getRelation(String relationName, - Optional filterExpression, - Optional sorting, - Optional pagination, - boolean checked) { - - if (checked && !checkRelation(relationName)) { + private Set getRelation(com.yahoo.elide.request.Relationship relationship, + boolean checked) { + if (checked && !checkRelation(relationship)) { return Collections.emptySet(); } - final Class relationClass = dictionary.getParameterizedType(obj, relationName); + final Class relationClass = dictionary.getParameterizedType(obj, relationship.getName()); + + Optional pagination = Optional.ofNullable(relationship.getProjection().getPagination()); + if (pagination.isPresent() && !pagination.get().isDefaultInstance() && !CanPaginateVisitor.canPaginate(relationClass, dictionary, requestScope)) { throw new InvalidPredicateException(String.format("Cannot paginate %s", dictionary.getJsonAliasFor(relationClass))); } - return getRelationUnchecked(relationName, filterExpression, sorting, pagination); + return getRelationUnchecked(relationship); } /** * Check the permissions of the relationship, and return true or false. - * @param relationName The relationship to the entity + * @param relationship The relationship to the entity * @return True if the relationship to the entity has valid permissions for the user */ - protected boolean checkRelation(String relationName) { - List relations = dictionary.getRelationships(obj); + protected boolean checkRelation(com.yahoo.elide.request.Relationship relationship) { + String relationName = relationship.getName(); String realName = dictionary.getNameFromAlias(obj, relationName); relationName = (realName == null) ? relationName : realName; - if (relationName == null || relations == null || !relations.contains(relationName)) { - throw new InvalidAttributeException(relationName, type); - } + assertRelationshipExists(relationName); checkFieldAwareDeferPermissions(ReadPermission.class, relationName, null, null); @@ -970,70 +1025,73 @@ protected boolean checkRelation(String relationName) { /** * Get collection of resources from relation field. * - * @param relationName field - * @param filterExpression An optional filter expression + * @param relationship the relationship to fetch * @return collection relation */ - protected Set getRelationChecked(String relationName, - Optional filterExpression, - Optional sorting, - Optional pagination) { - if (!checkRelation(relationName)) { + protected Set getRelationChecked(com.yahoo.elide.request.Relationship relationship) { + if (!checkRelation(relationship)) { return Collections.emptySet(); } - return getRelationUnchecked(relationName, filterExpression, sorting, pagination); + return getRelationUnchecked(relationship); } /** - * Retrieve an uncheck set of relations. - * - * @param relationName field - * @param filterExpression An optional filter expression - * @param sorting the sorting clause - * @param pagination the pagination params - * @return the resources in the relationship - */ - private Set getRelationUnchecked(String relationName, - Optional filterExpression, - Optional sorting, - Optional pagination) { + * Retrieve an unchecked set of relations. + */ + private Set getRelationUnchecked(com.yahoo.elide.request.Relationship relationship) { + String relationName = relationship.getName(); + FilterExpression filterExpression = relationship.getProjection().getFilterExpression(); + Pagination pagination = relationship.getProjection().getPagination(); + Sorting sorting = relationship.getProjection().getSorting(); + RelationshipType type = getRelationshipType(relationName); final Class relationClass = dictionary.getParameterizedType(obj, relationName); if (relationClass == null) { throw new InvalidAttributeException(relationName, this.getType()); } - Optional computedPagination = pagination.map(p -> p.evaluate(relationClass)); + Optional computedPagination = Optional.ofNullable(pagination) + .map(p -> p.evaluate(relationClass)); //Invoke filterExpressionCheck and then merge with filterExpression. Optional permissionFilter = getPermissionFilterExpression(relationClass, requestScope); - Optional computedFilters = filterExpression; + Optional computedFilters = Optional.ofNullable(filterExpression); - if (permissionFilter.isPresent() && filterExpression.isPresent()) { + if (permissionFilter.isPresent() && filterExpression != null) { FilterExpression mergedExpression = - new AndFilterExpression(filterExpression.get(), permissionFilter.get()); + new AndFilterExpression(filterExpression, permissionFilter.get()); computedFilters = Optional.of(mergedExpression); } else if (permissionFilter.isPresent()) { computedFilters = permissionFilter; } - Object val = transaction.getRelation(transaction, obj, relationName, - computedFilters, sorting, computedPagination, requestScope); + com.yahoo.elide.request.Relationship modifiedRelationship = relationship.copyOf() + .projection(relationship.getProjection().copyOf() + .filterExpression(computedFilters.orElse(null)) + .sorting(sorting) + .pagination(computedPagination.orElse(null)) + .build() + ).build(); + + Object val = transaction.getRelation(transaction, obj, modifiedRelationship, requestScope); if (val == null) { return Collections.emptySet(); } Set resources = Sets.newLinkedHashSet(); + if (val instanceof Iterable) { Iterable filteredVal = (Iterable) val; resources = new PersistentResourceSet(this, filteredVal, requestScope); } else if (type.isToOne()) { - resources = new SingleElementSet<>( - new PersistentResource<>(val, this, requestScope.getUUIDFor(val), requestScope)); + resources = new SingleElementSet( + new PersistentResource(val, this, + requestScope.getUUIDFor(val), requestScope)); } else { - resources.add(new PersistentResource<>(val, this, requestScope.getUUIDFor(val), requestScope)); + resources.add(new PersistentResource(val, this, + requestScope.getUUIDFor(val), requestScope)); } return resources; @@ -1072,10 +1130,20 @@ public RelationshipType getRelationshipType(String relation) { * @param attr Attribute name * @return Object value for attribute */ + @Deprecated public Object getAttribute(String attr) { return this.getValueChecked(attr); } + /** + * Get the value for a particular attribute (i.e. non-relational field) + * @param attr the Attribute + * @return Object value for attribute + */ + public Object getAttribute(Attribute attr) { + return this.getValueChecked(attr); + } + /** * Wrapped Entity bean. * @@ -1189,8 +1257,8 @@ public Resource toResource() { * Fetch a resource with support for lambda function for getting relationships and attributes. * @return The Resource */ - public Resource toResourceWithSortingAndPagination() { - return toResource(this::getRelationshipsWithSortingAndPagination, this::getAttributes); + public Resource toResource(EntityProjection projection) { + return toResource(() -> { return getRelationships(projection); }, this::getAttributes); } /** @@ -1199,7 +1267,7 @@ public Resource toResourceWithSortingAndPagination() { * @param attributeSupplier The attribute supplier * @return The Resource */ - public Resource toResource(final Supplier> relationshipSupplier, + private Resource toResource(final Supplier> relationshipSupplier, final Supplier> attributeSupplier) { final Resource resource = new Resource(type, (obj == null) ? uuid.orElseThrow( @@ -1217,8 +1285,17 @@ public Resource toResource(final Supplier> relationshi */ protected Map getRelationships() { return getRelationshipsWithRelationshipFunction((relationName) -> { - Optional filterExpression = requestScope.getExpressionForRelation(this, relationName); - return getRelationCheckedFiltered(relationName, filterExpression, Optional.empty(), Optional.empty()); + Optional filterExpression = requestScope.getExpressionForRelation(getResourceClass(), + relationName); + + return getRelationCheckedFiltered(com.yahoo.elide.request.Relationship.builder() + .alias(relationName) + .name(relationName) + .projection(EntityProjection.builder() + .type(dictionary.getParameterizedType(getResourceClass(), relationName)) + .filterExpression(filterExpression.orElse(null)) + .build()) + .build()); }); } @@ -1227,14 +1304,11 @@ protected Map getRelationships() { * * @return Relationship mapping */ - protected Map getRelationshipsWithSortingAndPagination() { - return getRelationshipsWithRelationshipFunction((relationName) -> { - Optional filterExpression = requestScope.getExpressionForRelation(this, relationName); - Optional sorting = Optional.ofNullable(requestScope.getSorting()); - Optional pagination = Optional.ofNullable(requestScope.getPagination()); - return getRelationCheckedFiltered(relationName, - filterExpression, sorting, pagination); - }); + private Map getRelationships(EntityProjection projection) { + return getRelationshipsWithRelationshipFunction( + (relationName) -> getRelationCheckedFiltered(projection.getRelationship(relationName) + .orElseThrow(IllegalStateException::new) + )); } /** @@ -1325,6 +1399,7 @@ protected void nullValue(String fieldName, PersistentResource oldValue) { * @param fieldName the field name * @return value value */ + @Deprecated protected Object getValueChecked(String fieldName) { requestScope.publishLifecycleEvent(this, CRUDEvent.CRUDAction.READ); requestScope.publishLifecycleEvent(this, fieldName, CRUDEvent.CRUDAction.READ, Optional.empty()); @@ -1332,6 +1407,18 @@ protected Object getValueChecked(String fieldName) { return getValue(getObject(), fieldName, requestScope); } + /** + * Gets a value from an entity and checks read permissions. + * @param attribute the attribute to fetch. + * @return value value + */ + protected Object getValueChecked(Attribute attribute) { + requestScope.publishLifecycleEvent(this, CRUDEvent.CRUDAction.READ); + requestScope.publishLifecycleEvent(this, attribute.getName(), CRUDEvent.CRUDAction.READ, Optional.empty()); + checkFieldAwareDeferPermissions(ReadPermission.class, attribute.getName(), (Object) null, (Object) null); + return transaction.getAttribute(getObject(), attribute, requestScope); + } + /** * Retrieve an object without checking read permissions (i.e. value is used internally and not sent to others) * @@ -1439,7 +1526,9 @@ protected void delFromCollection( */ protected void setValue(String fieldName, Object value) { final Object original = getValueUnchecked(fieldName); + dictionary.setValue(obj, fieldName, value); + triggerUpdate(fieldName, original, value); } @@ -1451,8 +1540,7 @@ protected void setValue(String fieldName, Object value) { * @return the value */ public static Object getValue(Object target, String fieldName, RequestScope requestScope) { - EntityDictionary dictionary = requestScope.getDictionary(); - return dictionary.getValue(target, fieldName, requestScope); + return requestScope.getDictionary().getValue(target, fieldName, requestScope); } /** @@ -1468,7 +1556,8 @@ protected void deleteInverseRelation(String relationName, Object inverseEntity) Class inverseType = dictionary.getType(inverseEntity.getClass(), inverseField); String uuid = requestScope.getUUIDFor(inverseEntity); - PersistentResource inverseResource = new PersistentResource(inverseEntity, this, uuid, requestScope); + PersistentResource inverseResource = new PersistentResource(inverseEntity, + this, uuid, requestScope); Object inverseRelation = inverseResource.getValueUnchecked(inverseField); if (inverseRelation == null) { diff --git a/elide-core/src/main/java/com/yahoo/elide/core/RequestScope.java b/elide-core/src/main/java/com/yahoo/elide/core/RequestScope.java index 1354e77b6c..a0ab953d20 100644 --- a/elide-core/src/main/java/com/yahoo/elide/core/RequestScope.java +++ b/elide-core/src/main/java/com/yahoo/elide/core/RequestScope.java @@ -29,6 +29,7 @@ import com.yahoo.elide.core.sort.Sorting; import com.yahoo.elide.jsonapi.JsonApiMapper; import com.yahoo.elide.jsonapi.models.JsonApiDocument; +import com.yahoo.elide.request.EntityProjection; import com.yahoo.elide.security.ChangeSpec; import com.yahoo.elide.security.PermissionExecutor; import com.yahoo.elide.security.User; @@ -38,6 +39,7 @@ import io.reactivex.subjects.PublishSubject; import io.reactivex.subjects.ReplaySubject; import lombok.Getter; +import lombok.Setter; import java.util.Collections; import java.util.HashMap; @@ -59,7 +61,7 @@ public class RequestScope implements com.yahoo.elide.security.RequestScope { @Getter private final JsonApiDocument jsonApiDocument; @Getter private final DataStoreTransaction transaction; @Getter private final User user; - @Getter private final EntityDictionary dictionary; + @Getter protected final EntityDictionary dictionary; @Getter private final JsonApiMapper mapper; @Getter private final AuditLogger auditLogger; @Getter private final Optional> queryParams; @@ -75,8 +77,11 @@ public class RequestScope implements com.yahoo.elide.security.RequestScope { @Getter private final ElideSettings elideSettings; @Getter private final boolean useFilterExpressions; @Getter private final int updateStatusCode; - @Getter private final MultipleFilterDialect filterDialect; + + //TODO - this ought to be read only and set in the constructor. + @Getter @Setter private EntityProjection entityProjection; + private final Map expressionsByType; private PublishSubject lifecycleEvents; @@ -232,7 +237,7 @@ public boolean isNewResource(Object entity) { * @param queryParams The request query parameters * @return Parsed sparseFields map */ - private static Map> parseSparseFields(MultivaluedMap queryParams) { + public static Map> parseSparseFields(MultivaluedMap queryParams) { Map> result = new HashMap<>(); for (Map.Entry> kv : queryParams.entrySet()) { @@ -263,6 +268,15 @@ public Optional getFilterExpressionByType(String type) { return Optional.ofNullable(expressionsByType.get(type)); } + /** + * Get filter expression for a specific collection type. + * @param entityClass The class to lookup + * @return The filter expression for the given type + */ + public Optional getFilterExpressionByType(Class entityClass) { + return Optional.ofNullable(expressionsByType.get(dictionary.getJsonAliasFor(entityClass))); + } + /** * Get the global/cross-type filter expression. * @param loadClass Entity class @@ -281,14 +295,14 @@ public Optional getLoadFilterExpression(Class loadClass) { /** * Get the filter expression for a particular relationship - * @param parent The object which has the relationship + * @param parentType The parent type which has the relationship * @param relationName The relationship name * @return A type specific filter expression for the given relationship */ - public Optional getExpressionForRelation(PersistentResource parent, String relationName) { - final Class entityClass = dictionary.getParameterizedType(parent.getObject(), relationName); + public Optional getExpressionForRelation(Class parentType, String relationName) { + final Class entityClass = dictionary.getParameterizedType(parentType, relationName); if (entityClass == null) { - throw new InvalidAttributeException(relationName, parent.getType()); + throw new InvalidAttributeException(relationName, dictionary.getJsonAliasFor(parentType)); } if (dictionary.isMappedInterface(entityClass) && interfaceHasFilterExpression(entityClass)) { throw new InvalidOperationException( diff --git a/elide-core/src/main/java/com/yahoo/elide/core/TimedFunction.java b/elide-core/src/main/java/com/yahoo/elide/core/TimedFunction.java new file mode 100644 index 0000000000..df5b187b31 --- /dev/null +++ b/elide-core/src/main/java/com/yahoo/elide/core/TimedFunction.java @@ -0,0 +1,40 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ + +package com.yahoo.elide.core; + +import lombok.Data; +import lombok.extern.slf4j.Slf4j; + +import java.util.function.Supplier; + +/** + * Wraps a function and logs how long it took to run (in millis). + * @param The function return type. + */ +@Slf4j +@Data +public class TimedFunction implements Supplier { + + public TimedFunction(Supplier toRun, String logMessage) { + this.toRun = toRun; + this.logMessage = logMessage; + } + + private Supplier toRun; + private String logMessage; + + @Override + public R get() { + long start = System.currentTimeMillis(); + R ret = toRun.get(); + long end = System.currentTimeMillis(); + + log.debug(logMessage + "\tTime spent: {}", end - start); + + return ret; + } +} diff --git a/elide-core/src/main/java/com/yahoo/elide/core/datastore/inmemory/HashMapDataStore.java b/elide-core/src/main/java/com/yahoo/elide/core/datastore/inmemory/HashMapDataStore.java index b20183c4d6..8d2f72fd82 100644 --- a/elide-core/src/main/java/com/yahoo/elide/core/datastore/inmemory/HashMapDataStore.java +++ b/elide-core/src/main/java/com/yahoo/elide/core/datastore/inmemory/HashMapDataStore.java @@ -27,8 +27,8 @@ * Simple in-memory only database. */ public class HashMapDataStore implements DataStore, DataStoreTestHarness { - private final Map, Map> dataStore = Collections.synchronizedMap(new HashMap<>()); - @Getter private EntityDictionary dictionary; + protected final Map, Map> dataStore = Collections.synchronizedMap(new HashMap<>()); + @Getter protected EntityDictionary dictionary; @Getter private final Set beanPackages; @Getter private final ConcurrentHashMap, AtomicLong> typeIds = new ConcurrentHashMap<>(); diff --git a/elide-core/src/main/java/com/yahoo/elide/core/datastore/inmemory/HashMapStoreTransaction.java b/elide-core/src/main/java/com/yahoo/elide/core/datastore/inmemory/HashMapStoreTransaction.java index 943ca2d8bd..a1ba45304d 100644 --- a/elide-core/src/main/java/com/yahoo/elide/core/datastore/inmemory/HashMapStoreTransaction.java +++ b/elide-core/src/main/java/com/yahoo/elide/core/datastore/inmemory/HashMapStoreTransaction.java @@ -10,15 +10,17 @@ import com.yahoo.elide.core.RequestScope; import com.yahoo.elide.core.exceptions.TransactionException; import com.yahoo.elide.core.filter.expression.FilterExpression; -import com.yahoo.elide.core.pagination.Pagination; import com.yahoo.elide.core.sort.Sorting; +import com.yahoo.elide.request.EntityProjection; +import com.yahoo.elide.request.Relationship; +import lombok.extern.slf4j.Slf4j; + import java.io.IOException; import java.io.Serializable; import java.util.ArrayList; import java.util.List; import java.util.Map; -import java.util.Optional; import java.util.concurrent.atomic.AtomicLong; import javax.persistence.GeneratedValue; @@ -131,31 +133,25 @@ public void setId(Object value, String id) { @Override public Object getRelation(DataStoreTransaction relationTx, Object entity, - String relationName, - Optional filterExpression, - Optional sorting, - Optional pagination, + Relationship relationship, RequestScope scope) { - return dictionary.getValue(entity, relationName, scope); + return dictionary.getValue(entity, relationship.getName(), scope); } @Override - public Iterable loadObjects(Class entityClass, Optional filterExpression, - Optional sorting, Optional pagination, + public Iterable loadObjects(EntityProjection projection, RequestScope scope) { synchronized (dataStore) { - Map data = dataStore.get(entityClass); + Map data = dataStore.get(projection.getType()); return data.values(); } } @Override - public Object loadObject(Class entityClass, Serializable id, - Optional filterExpression, - RequestScope scope) { + public Object loadObject(EntityProjection projection, Serializable id, RequestScope scope) { synchronized (dataStore) { - Map data = dataStore.get(entityClass); + Map data = dataStore.get(projection.getType()); if (data == null) { return null; } diff --git a/elide-core/src/main/java/com/yahoo/elide/core/datastore/inmemory/InMemoryStoreTransaction.java b/elide-core/src/main/java/com/yahoo/elide/core/datastore/inmemory/InMemoryStoreTransaction.java index 0c4475a3c2..3eba0004b9 100644 --- a/elide-core/src/main/java/com/yahoo/elide/core/datastore/inmemory/InMemoryStoreTransaction.java +++ b/elide-core/src/main/java/com/yahoo/elide/core/datastore/inmemory/InMemoryStoreTransaction.java @@ -17,6 +17,9 @@ import com.yahoo.elide.core.filter.expression.InMemoryFilterExecutor; import com.yahoo.elide.core.pagination.Pagination; import com.yahoo.elide.core.sort.Sorting; +import com.yahoo.elide.request.Attribute; +import com.yahoo.elide.request.EntityProjection; +import com.yahoo.elide.request.Relationship; import com.yahoo.elide.security.User; import org.apache.commons.lang3.tuple.Pair; @@ -39,6 +42,8 @@ */ public class InMemoryStoreTransaction implements DataStoreTransaction { + private final DataStoreTransaction tx; + private static final Comparator NULL_SAFE_COMPARE = (a, b) -> { if (a == null && b == null) { return 0; @@ -53,8 +58,6 @@ public class InMemoryStoreTransaction implements DataStoreTransaction { } }; - private DataStoreTransaction tx; - /** * Fetches data from the store. */ @@ -71,6 +74,80 @@ public InMemoryStoreTransaction(DataStoreTransaction tx) { this.tx = tx; } + @Override + public Object getRelation(DataStoreTransaction relationTx, + Object entity, + Relationship relationship, + RequestScope scope) { + DataFetcher fetcher = new DataFetcher() { + @Override + public Object fetch(Optional filterExpression, + Optional sorting, + Optional pagination, + RequestScope scope) { + + return tx.getRelation(relationTx, entity, relationship.copyOf() + .projection(relationship.getProjection().copyOf() + .filterExpression(filterExpression.orElse(null)) + .sorting(sorting.orElse(null)) + .pagination(pagination.orElse(null)) + .build() + ).build(), scope); + } + }; + + + /* + * If we are mutating multiple entities, the data store transaction cannot perform filter & pagination directly. + * It must be done in memory by Elide as some newly created entities have not yet been persisted. + */ + boolean filterInMemory = scope.getNewPersistentResources().size() > 0; + return fetchData(fetcher, relationship.getProjection().getType(), + Optional.ofNullable(relationship.getProjection().getFilterExpression()), + Optional.ofNullable(relationship.getProjection().getSorting()), + Optional.ofNullable(relationship.getProjection().getPagination()), + filterInMemory, scope); + } + + @Override + public Object loadObject(EntityProjection projection, + Serializable id, + RequestScope scope) { + + if (projection.getFilterExpression() == null + || tx.supportsFiltering(projection.getType(), + projection.getFilterExpression()) == FeatureSupport.FULL) { + return tx.loadObject(projection, id, scope); + } else { + return DataStoreTransaction.super.loadObject(projection, id, scope); + } + } + + @Override + public Iterable loadObjects(EntityProjection projection, + RequestScope scope) { + + DataFetcher fetcher = new DataFetcher() { + @Override + public Iterable fetch(Optional filterExpression, + Optional sorting, + Optional pagination, + RequestScope scope) { + + return tx.loadObjects(projection.copyOf() + .filterExpression(filterExpression.orElse(null)) + .pagination(pagination.orElse(null)) + .sorting(sorting.orElse(null)) + .build(), scope); + } + }; + + return (Iterable) fetchData(fetcher, projection.getType(), + Optional.ofNullable(projection.getFilterExpression()), + Optional.ofNullable(projection.getSorting()), + Optional.ofNullable(projection.getPagination()), + false, scope); + } @Override public void save(Object entity, RequestScope scope) { @@ -98,34 +175,8 @@ public T createNewObject(Class entityClass) { } @Override - public Object getRelation(DataStoreTransaction relationTx, - Object entity, - String relationName, - Optional filterExpression, - Optional sorting, - Optional pagination, - RequestScope scope) { - - Class relationClass = scope.getDictionary().getParameterizedType(entity, relationName); - - DataFetcher fetcher = new DataFetcher() { - @Override - public Object fetch(Optional filterExpression, - Optional sorting, - Optional pagination, - RequestScope scope) { - - return tx.getRelation(relationTx, entity, relationName, filterExpression, sorting, pagination, scope); - } - }; - - - /* - * If we are mutating multiple entities, the data store transaction cannot perform filter & pagination directly. - * It must be done in memory by Elide as some newly created entities have not yet been persisted. - */ - boolean filterInMemory = scope.getNewPersistentResources().size() > 0; - return fetchData(fetcher, relationClass, filterExpression, sorting, pagination, filterInMemory, scope); + public void close() throws IOException { + tx.close(); } @Override @@ -148,14 +199,13 @@ public void updateToOneRelation(DataStoreTransaction relationTx, } @Override - public Object getAttribute(Object entity, String attributeName, RequestScope scope) { - return tx.getAttribute(entity, attributeName, scope); + public Object getAttribute(Object entity, Attribute attribute, RequestScope scope) { + return tx.getAttribute(entity, attribute, scope); } @Override - public void setAttribute(Object entity, String attributeName, Object attributeValue, RequestScope scope) { - tx.setAttribute(entity, attributeName, attributeValue, scope); - + public void setAttribute(Object entity, Attribute attribute, RequestScope scope) { + tx.setAttribute(entity, attribute, scope); } @Override @@ -173,45 +223,6 @@ public void createObject(Object entity, RequestScope scope) { tx.createObject(entity, scope); } - @Override - public Object loadObject(Class entityClass, - Serializable id, - Optional filterExpression, - RequestScope scope) { - - if (! filterExpression.isPresent() - || tx.supportsFiltering(entityClass, filterExpression.get()) == FeatureSupport.FULL) { - return tx.loadObject(entityClass, id, filterExpression, scope); - } - return DataStoreTransaction.super.loadObject(entityClass, id, filterExpression, scope); - } - - @Override - public Iterable loadObjects(Class entityClass, - Optional filterExpression, - Optional sorting, - Optional pagination, - RequestScope scope) { - - DataFetcher fetcher = new DataFetcher() { - @Override - public Iterable fetch(Optional filterExpression, - Optional sorting, - Optional pagination, - RequestScope scope) { - return tx.loadObjects(entityClass, filterExpression, sorting, pagination, scope); - } - }; - - return (Iterable) fetchData(fetcher, entityClass, - filterExpression, sorting, pagination, false, scope); - } - - @Override - public void close() throws IOException { - tx.close(); - } - private Iterable filterLoadedData(Iterable loadedRecords, Optional filterExpression, RequestScope scope) { diff --git a/elide-core/src/main/java/com/yahoo/elide/core/datastore/wrapped/TransactionWrapper.java b/elide-core/src/main/java/com/yahoo/elide/core/datastore/wrapped/TransactionWrapper.java index f27e4e0219..3171047efc 100644 --- a/elide-core/src/main/java/com/yahoo/elide/core/datastore/wrapped/TransactionWrapper.java +++ b/elide-core/src/main/java/com/yahoo/elide/core/datastore/wrapped/TransactionWrapper.java @@ -9,15 +9,16 @@ import com.yahoo.elide.core.DataStoreTransaction; import com.yahoo.elide.core.RequestScope; import com.yahoo.elide.core.filter.expression.FilterExpression; -import com.yahoo.elide.core.pagination.Pagination; import com.yahoo.elide.core.sort.Sorting; +import com.yahoo.elide.request.Attribute; +import com.yahoo.elide.request.EntityProjection; +import com.yahoo.elide.request.Relationship; import com.yahoo.elide.security.User; import lombok.AllArgsConstructor; import lombok.Data; import java.io.IOException; import java.io.Serializable; -import java.util.Optional; import java.util.Set; /** @@ -44,16 +45,15 @@ public T createNewObject(Class entityClass) { } @Override - public Object loadObject(Class entityClass, Serializable id, Optional filterExpression, + public Object loadObject(EntityProjection projection, Serializable id, RequestScope scope) { - return tx.loadObject(entityClass, id, filterExpression, scope); + return tx.loadObject(projection, id, scope); } @Override - public Object getRelation(DataStoreTransaction relationTx, Object entity, String relationName, - Optional filterExpression, Optional sorting, - Optional pagination, RequestScope scope) { - return tx.getRelation(relationTx, entity, relationName, filterExpression, sorting, pagination, scope); + public Object getRelation(DataStoreTransaction relationTx, Object entity, + Relationship relationship, RequestScope scope) { + return tx.getRelation(relationTx, entity, relationship, scope); } @Override @@ -71,13 +71,13 @@ public void updateToOneRelation(DataStoreTransaction relationTx, Object entity, } @Override - public Object getAttribute(Object entity, String attributeName, RequestScope scope) { - return tx.getAttribute(entity, attributeName, scope); + public Object getAttribute(Object entity, Attribute attribute, RequestScope scope) { + return tx.getAttribute(entity, attribute, scope); } @Override - public void setAttribute(Object entity, String attributeName, Object attributeValue, RequestScope scope) { - tx.setAttribute(entity, attributeName, attributeValue, scope); + public void setAttribute(Object entity, Attribute attribute, RequestScope scope) { + tx.setAttribute(entity, attribute, scope); } @Override @@ -119,16 +119,11 @@ public void commit(RequestScope requestScope) { @Override public void createObject(Object o, RequestScope requestScope) { tx.createObject(o, requestScope); - } @Override - public Iterable loadObjects(Class entityClass, - Optional filterExpression, - Optional sorting, - Optional pagination, - RequestScope requestScope) { - return tx.loadObjects(entityClass, filterExpression, sorting, pagination, requestScope); + public Iterable loadObjects(EntityProjection projection, RequestScope scope) { + return tx.loadObjects(projection, scope); } @Override diff --git a/elide-core/src/main/java/com/yahoo/elide/core/filter/FilterPredicate.java b/elide-core/src/main/java/com/yahoo/elide/core/filter/FilterPredicate.java index 3b7be39375..70446b5f19 100644 --- a/elide-core/src/main/java/com/yahoo/elide/core/filter/FilterPredicate.java +++ b/elide-core/src/main/java/com/yahoo/elide/core/filter/FilterPredicate.java @@ -105,21 +105,34 @@ public FilterPredicate scopedBy(PathElement scope) { } /** - * Returns an alias that uniquely identifies the last collection of entities in the path. - * @return An alias for the path. + * Generate alias for representing a relationship path which dose not include the last field name. + * The path would start with the class alias of the first element, and then each field would append "_fieldName" to + * the result. + * The last field would not be included as that's not a part of the relationship path. + * + * @param path path that represents a relationship chain + * @return relationship path alias, i.e. foo.bar.baz would be foo_bar */ - public String getAlias() { - List elements = path.getPathElements(); + public static String getPathAlias(Path path) { + List elements = path.getPathElements(); + String alias = getTypeAlias(elements.get(0).getType()); - PathElement last = elements.get(elements.size() - 1); - - if (elements.size() == 1) { - return getTypeAlias(last.getType()); + for (int i = 0; i < elements.size() - 1; i++) { + alias = appendAlias(alias, elements.get(i).getFieldName()); } - PathElement previous = elements.get(elements.size() - 2); + return alias; + } - return getTypeAlias(previous.getType()) + UNDERSCORE + previous.getFieldName(); + /** + * Append a new field to a parent alias to get new alias. + * + * @param parentAlias parent path alias + * @param fieldName field name + * @return alias for the field + */ + public static String appendAlias(String parentAlias, String fieldName) { + return parentAlias + "_" + fieldName; } /** diff --git a/elide-core/src/main/java/com/yahoo/elide/core/filter/dialect/RSQLFilterDialect.java b/elide-core/src/main/java/com/yahoo/elide/core/filter/dialect/RSQLFilterDialect.java index 68e5a55037..3832a1d3ff 100644 --- a/elide-core/src/main/java/com/yahoo/elide/core/filter/dialect/RSQLFilterDialect.java +++ b/elide-core/src/main/java/com/yahoo/elide/core/filter/dialect/RSQLFilterDialect.java @@ -168,7 +168,6 @@ public Map parseTypedExpression(String path, Multivalu return expressionByType; } - /** * Parses a RSQL string into an Elide FilterExpression. * @param expressionText the RSQL string diff --git a/elide-core/src/main/java/com/yahoo/elide/core/filter/expression/AndFilterExpression.java b/elide-core/src/main/java/com/yahoo/elide/core/filter/expression/AndFilterExpression.java index 9571351ba3..4c87ef7a42 100644 --- a/elide-core/src/main/java/com/yahoo/elide/core/filter/expression/AndFilterExpression.java +++ b/elide-core/src/main/java/com/yahoo/elide/core/filter/expression/AndFilterExpression.java @@ -17,6 +17,32 @@ public class AndFilterExpression implements FilterExpression { @Getter private FilterExpression left; @Getter private FilterExpression right; + /** + * Returns a new {@link AndFilterExpression} instance with the specified null-able left and right operands. + *

+ * The publication rules are + *

    + *
  1. If both left and right are not {@code null}, this method produces the same instance as + * {@link #AndFilterExpression(FilterExpression, FilterExpression)} does, + *
  2. If one of them is {@code null}, the other non-null is returned with no modification, + *
  3. If both left and right are {@code null}, this method returns + * {@code null}. + *
+ * + * @param left The provided left {@link FilterExpression} + * @param right The provided right {@link FilterExpression} + * + * @return a new {@link AndFilterExpression} instance or {@code null} + */ + public static FilterExpression fromPair(FilterExpression left, FilterExpression right) { + if (left != null && right != null) { + return new AndFilterExpression(left, right); + } else if (left == null) { + return right; + } + return left; + } + public AndFilterExpression(FilterExpression left, FilterExpression right) { this.left = left; this.right = right; diff --git a/elide-core/src/main/java/com/yahoo/elide/core/filter/expression/OrFilterExpression.java b/elide-core/src/main/java/com/yahoo/elide/core/filter/expression/OrFilterExpression.java index 02cfb62ed8..90f5e628ef 100644 --- a/elide-core/src/main/java/com/yahoo/elide/core/filter/expression/OrFilterExpression.java +++ b/elide-core/src/main/java/com/yahoo/elide/core/filter/expression/OrFilterExpression.java @@ -18,6 +18,32 @@ public class OrFilterExpression implements FilterExpression { @Getter private FilterExpression left; @Getter private FilterExpression right; + /** + * Returns a new {@link OrFilterExpression} instance with the specified null-able left and right operands. + *

+ * The publication rules are + *

    + *
  1. If both left and right are not {@code null}, this method produces the same instance as + * {@link #OrFilterExpression(FilterExpression, FilterExpression)} does, + *
  2. If one of them is {@code null}, the other non-null is returned with no modification, + *
  3. If both left and right are {@code null}, this method returns + * {@code null}. + *
+ * + * @param left The provided left {@link FilterExpression} + * @param right The provided right {@link FilterExpression} + * + * @return a new {@link OrFilterExpression} instance or {@code null} + */ + public static FilterExpression fromPair(FilterExpression left, FilterExpression right) { + if (left != null && right != null) { + return new OrFilterExpression(left, right); + } else if (left == null) { + return right; + } + return left; + } + public OrFilterExpression(FilterExpression left, FilterExpression right) { this.left = left; this.right = right; diff --git a/elide-core/src/main/java/com/yahoo/elide/core/pagination/Pagination.java b/elide-core/src/main/java/com/yahoo/elide/core/pagination/Pagination.java index 61ae0b6241..860146762e 100644 --- a/elide-core/src/main/java/com/yahoo/elide/core/pagination/Pagination.java +++ b/elide-core/src/main/java/com/yahoo/elide/core/pagination/Pagination.java @@ -85,6 +85,21 @@ private Pagination(Map pageData, int defaultMaxPageSize, this.defaultPageSize = defaultPageSize; } + /** + * Set limit. + * + * @param perPage page size. + */ + public void setLimit(Integer perPage) { + this.limit = perPage; + pageData.put(PaginationKey.limit, perPage); + } + + public void setOffset(Integer offset) { + this.offset = offset; + pageData.put(PaginationKey.offset, offset); + } + /** * TODO - Refactor Pagination. * IMPORTANT - This method should only be used for testing until Pagination is refactored. The @@ -99,8 +114,8 @@ private Pagination(Map pageData, int defaultMaxPageSize, public static Pagination fromOffsetAndLimit(int limit, int offset, boolean generatePageTotals) { ImmutableMap.Builder pageData = ImmutableMap.builder() - .put(PAGE_KEYS.get(PAGE_OFFSET_KEY), offset) - .put(PAGE_KEYS.get(PAGE_LIMIT_KEY), limit); + .put(PAGE_KEYS.get(PAGE_OFFSET_KEY), offset) + .put(PAGE_KEYS.get(PAGE_LIMIT_KEY), limit); if (generatePageTotals) { pageData.put(PAGE_KEYS.get(PAGE_TOTALS_KEY), 1); diff --git a/elide-core/src/main/java/com/yahoo/elide/core/sort/Sorting.java b/elide-core/src/main/java/com/yahoo/elide/core/sort/Sorting.java index 042f80e5d7..ec1d091417 100644 --- a/elide-core/src/main/java/com/yahoo/elide/core/sort/Sorting.java +++ b/elide-core/src/main/java/com/yahoo/elide/core/sort/Sorting.java @@ -9,6 +9,7 @@ import com.yahoo.elide.core.Path; import com.yahoo.elide.core.exceptions.InvalidValueException; +import lombok.EqualsAndHashCode; import lombok.ToString; import java.util.Arrays; @@ -23,6 +24,7 @@ * Generates a simple wrapper around the sort fields from the JSON-API GET Query. */ @ToString +@EqualsAndHashCode public class Sorting { /** diff --git a/elide-core/src/main/java/com/yahoo/elide/extensions/PatchRequestScope.java b/elide-core/src/main/java/com/yahoo/elide/extensions/PatchRequestScope.java index 4e1e993cd2..1333e10162 100644 --- a/elide-core/src/main/java/com/yahoo/elide/extensions/PatchRequestScope.java +++ b/elide-core/src/main/java/com/yahoo/elide/extensions/PatchRequestScope.java @@ -8,6 +8,7 @@ import com.yahoo.elide.ElideSettings; import com.yahoo.elide.core.DataStoreTransaction; import com.yahoo.elide.core.RequestScope; +import com.yahoo.elide.jsonapi.EntityProjectionMaker; import com.yahoo.elide.jsonapi.models.JsonApiDocument; import com.yahoo.elide.security.User; @@ -50,5 +51,6 @@ public PatchRequestScope( */ public PatchRequestScope(String path, JsonApiDocument jsonApiDocument, PatchRequestScope scope) { super(path, jsonApiDocument, scope); + this.setEntityProjection(new EntityProjectionMaker(dictionary, this).parsePath(path)); } } diff --git a/elide-core/src/main/java/com/yahoo/elide/jsonapi/EntityProjectionMaker.java b/elide-core/src/main/java/com/yahoo/elide/jsonapi/EntityProjectionMaker.java new file mode 100644 index 0000000000..b3e4eee9c5 --- /dev/null +++ b/elide-core/src/main/java/com/yahoo/elide/jsonapi/EntityProjectionMaker.java @@ -0,0 +1,383 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ + +package com.yahoo.elide.jsonapi; + +import com.yahoo.elide.core.EntityDictionary; +import com.yahoo.elide.core.Path; +import com.yahoo.elide.core.RequestScope; +import com.yahoo.elide.core.exceptions.InvalidCollectionException; +import com.yahoo.elide.core.filter.expression.FilterExpression; +import com.yahoo.elide.generated.parsers.CoreBaseVisitor; +import com.yahoo.elide.generated.parsers.CoreParser; +import com.yahoo.elide.parsers.JsonApiParser; +import com.yahoo.elide.request.Attribute; +import com.yahoo.elide.request.EntityProjection; +import com.yahoo.elide.request.Relationship; + +import com.google.common.collect.Sets; +import org.apache.commons.lang3.tuple.Pair; + +import lombok.Builder; +import lombok.Data; + +import java.util.Arrays; +import java.util.HashSet; +import java.util.Map; +import java.util.Set; +import java.util.function.Function; +import java.util.stream.Collectors; +import java.util.stream.Stream; + +import javax.ws.rs.core.MultivaluedHashMap; +import javax.ws.rs.core.MultivaluedMap; + +/** + * Converts a JSON-API request (URL and query parameters) into an EntityProjection. + */ +public class EntityProjectionMaker + extends CoreBaseVisitor, EntityProjectionMaker.NamedEntityProjection>> { + + /** + * An entity projection labeled with the class name or relationship name it is associated with. + */ + @Data + @Builder + public static class NamedEntityProjection { + private String name; + private EntityProjection projection; + } + + private static final String INCLUDE = "include"; + + private EntityDictionary dictionary; + private MultivaluedMap queryParams; + private Map> sparseFields; + private RequestScope scope; + + public EntityProjectionMaker(EntityDictionary dictionary, RequestScope scope) { + this.dictionary = dictionary; + this.queryParams = scope.getQueryParams().orElse(new MultivaluedHashMap<>()); + sparseFields = RequestScope.parseSparseFields(queryParams); + this.scope = scope; + } + + public EntityProjection parsePath(String path) { + return visit(JsonApiParser.parse(path)).apply(null).projection; + } + + public EntityProjection parseInclude(Class entityClass) { + return EntityProjection.builder() + .type(entityClass) + .relationships(toRelationshipSet(getIncludedRelationships(entityClass))) + .build(); + } + + @Override + public Function, NamedEntityProjection> visitRootCollectionLoadEntities( + CoreParser.RootCollectionLoadEntitiesContext ctx) { + return visitTerminalCollection(ctx.term()); + } + + @Override + public Function, NamedEntityProjection> visitSubCollectionReadCollection( + CoreParser.SubCollectionReadCollectionContext ctx) { + return visitTerminalCollection(ctx.term()); + } + + @Override + public Function, NamedEntityProjection> visitRootCollectionSubCollection( + CoreParser.RootCollectionSubCollectionContext ctx) { + return visitEntityWithSubCollection(ctx.entity(), ctx.subCollection()); + } + + @Override + public Function, NamedEntityProjection> visitSubCollectionSubCollection( + CoreParser.SubCollectionSubCollectionContext ctx) { + return visitEntityWithSubCollection(ctx.entity(), ctx.subCollection()); + } + + @Override + public Function, NamedEntityProjection> visitRootCollectionRelationship( + CoreParser.RootCollectionRelationshipContext ctx) { + return visitEntityWithRelationship(ctx.entity(), ctx.relationship()); + } + + @Override + public Function, NamedEntityProjection> visitSubCollectionRelationship( + CoreParser.SubCollectionRelationshipContext ctx) { + return visitEntityWithRelationship(ctx.entity(), ctx.relationship()); + } + + @Override + public Function, NamedEntityProjection> visitRootCollectionLoadEntity( + CoreParser.RootCollectionLoadEntityContext ctx) { + return (unused) -> { + return ctx.entity().accept(this).apply(null); + }; + } + + @Override + public Function, NamedEntityProjection> visitSubCollectionReadEntity( + CoreParser.SubCollectionReadEntityContext ctx) { + return (parentClass) -> { + return ctx.entity().accept(this).apply(parentClass); + }; + } + + @Override + public Function, NamedEntityProjection> visitRelationship(CoreParser.RelationshipContext ctx) { + return (parentClass) -> { + String entityName = ctx.term().getText(); + + Class entityClass = getEntityClass(parentClass, entityName); + FilterExpression filter = scope.getExpressionForRelation(parentClass, entityName).orElse(null); + + return NamedEntityProjection.builder() + .name(entityName) + .projection(EntityProjection.builder() + .filterExpression(filter) + .sorting(scope.getSorting()) + .pagination(scope.getPagination()) + .type(entityClass) + .build() + ).build(); + }; + } + + @Override + public Function, NamedEntityProjection> visitEntity(CoreParser.EntityContext ctx) { + return (parentClass) -> { + String entityName = ctx.term().getText(); + + Class entityClass = getEntityClass(parentClass, entityName); + + return NamedEntityProjection.builder() + .name(entityName) + .projection(EntityProjection.builder() + .type(entityClass) + .attributes(getSparseAttributes(entityClass)) + .relationships(toRelationshipSet(getRequiredRelationships(entityClass))) + .build() + ).build(); + }; + } + + @Override + protected Function, NamedEntityProjection> aggregateResult( + Function, NamedEntityProjection> aggregate, + Function, NamedEntityProjection> nextResult) { + + if (aggregate == null) { + return nextResult; + } else { + return aggregate; + } + } + + public EntityProjection visitIncludePath(Path path) { + Path.PathElement pathElement = path.getPathElements().get(0); + int size = path.getPathElements().size(); + + Class entityClass = pathElement.getFieldType(); + + if (size > 1) { + Path nextPath = new Path(path.getPathElements().subList(1, size)); + EntityProjection relationshipProjection = visitIncludePath(nextPath); + + return EntityProjection.builder() + .relationships(toRelationshipSet(getSparseRelationships(entityClass))) + .relationship(nextPath.getPathElements().get(0).getFieldName(), relationshipProjection) + .attributes(getSparseAttributes(entityClass)) + .filterExpression(scope.getFilterExpressionByType(entityClass).orElse(null)) + .type(entityClass) + .build(); + } + + return EntityProjection.builder() + .relationships(toRelationshipSet(getSparseRelationships(entityClass))) + .attributes(getSparseAttributes(entityClass)) + .type(entityClass) + .filterExpression(scope.getFilterExpressionByType(entityClass).orElse(null)) + .build(); + } + + private Function, NamedEntityProjection> visitEntityWithSubCollection(CoreParser.EntityContext entity, + CoreParser.SubCollectionContext subCollection) { + return (parentClass) -> { + String entityName = entity.term().getText(); + + Class entityClass = getEntityClass(parentClass, entityName); + + NamedEntityProjection projection = subCollection.accept(this).apply(entityClass); + + return NamedEntityProjection.builder() + .name(entityName) + .projection(EntityProjection.builder() + .type(entityClass) + .relationship(projection.name, projection.projection) + .build() + ).build(); + }; + } + + private Function, NamedEntityProjection> visitEntityWithRelationship(CoreParser.EntityContext entity, + CoreParser.RelationshipContext relationship) { + return (parentClass) -> { + String entityName = entity.term().getText(); + + Class entityClass = getEntityClass(parentClass, entityName); + + String relationshipName = relationship.term().getText(); + NamedEntityProjection relationshipProjection = relationship.accept(this).apply(entityClass); + + FilterExpression filter = scope.getFilterExpressionByType(entityClass).orElse(null); + + return NamedEntityProjection.builder() + .name(entityName) + .projection(EntityProjection.builder() + .type(entityClass) + .filterExpression(filter) + .relationships(toRelationshipSet(getRequiredRelationships(entityClass))) + .relationship(relationshipName, relationshipProjection.projection) + .build() + ).build(); + }; + } + + private Function, NamedEntityProjection> visitTerminalCollection(CoreParser.TermContext collectionName) { + return (parentClass) -> { + String collectionNameText = collectionName.getText(); + + Class entityClass = getEntityClass(parentClass, collectionNameText); + + FilterExpression filter; + if (parentClass == null) { + filter = scope.getLoadFilterExpression(entityClass).orElse(null); + } else { + filter = scope.getExpressionForRelation(parentClass, collectionNameText).orElse(null); + } + + return NamedEntityProjection.builder() + .name(collectionNameText) + .projection(EntityProjection.builder() + .filterExpression(filter) + .sorting(scope.getSorting()) + .pagination(scope.getPagination()) + .relationships(toRelationshipSet(getRequiredRelationships(entityClass))) + .attributes(getSparseAttributes(entityClass)) + .type(entityClass) + .build() + ).build(); + }; + } + + private Class getEntityClass(Class parentClass, String entityLabel) { + + //entityLabel represents a root collection. + if (parentClass == null) { + Class entityClass = dictionary.getEntityClass(entityLabel); + + if (entityClass != null) { + return entityClass; + } + + + //entityLabel represents a relationship. + } else if (dictionary.isRelation(parentClass, entityLabel)) { + return dictionary.getParameterizedType(parentClass, entityLabel); + } + + throw new InvalidCollectionException(entityLabel); + } + + private Map getIncludedRelationships(Class entityClass) { + Set includePaths = getIncludePaths(entityClass); + + Map relationships = includePaths.stream() + .map((path) -> Pair.of(path.getPathElements().get(0).getFieldName(), visitIncludePath(path))) + .collect(Collectors.toMap( + Pair::getKey, + Pair::getValue, + EntityProjection::merge + )); + + return relationships; + } + + private Set getSparseAttributes(Class entityClass) { + Set allAttributes = new HashSet<>(dictionary.getAttributes(entityClass)); + + Set sparseFieldsForEntity = sparseFields.get(dictionary.getJsonAliasFor(entityClass)); + if (sparseFieldsForEntity == null || sparseFieldsForEntity.isEmpty()) { + sparseFieldsForEntity = allAttributes; + } + + return Sets.intersection(allAttributes, sparseFieldsForEntity).stream() + .map(attributeName -> Attribute.builder() + .name(attributeName) + .type(dictionary.getParameterizedType(entityClass, attributeName)) + .build()) + .collect(Collectors.toSet()); + } + + private Map getSparseRelationships(Class entityClass) { + Set allRelationships = new HashSet<>(dictionary.getRelationships(entityClass)); + Set sparseFieldsForEntity = sparseFields.get(dictionary.getJsonAliasFor(entityClass)); + + if (sparseFieldsForEntity == null || sparseFieldsForEntity.isEmpty()) { + sparseFieldsForEntity = allRelationships; + } + + sparseFieldsForEntity = Sets.intersection(allRelationships, sparseFieldsForEntity); + + return sparseFieldsForEntity.stream() + .collect(Collectors.toMap( + Function.identity(), + (relationshipName) -> { + FilterExpression filter = scope.getExpressionForRelation(entityClass, relationshipName) + .orElse(null); + + return EntityProjection.builder() + .type(dictionary.getParameterizedType(entityClass, relationshipName)) + .filterExpression(filter) + .build(); + } + )); + } + + private Map getRequiredRelationships(Class entityClass) { + return Stream.concat( + getIncludedRelationships(entityClass).entrySet().stream(), + getSparseRelationships(entityClass).entrySet().stream() + ).collect(Collectors.toMap( + Map.Entry::getKey, + Map.Entry::getValue, + EntityProjection::merge + )); + } + + private Set getIncludePaths(Class entityClass) { + if (queryParams.get(INCLUDE) != null) { + return queryParams.get(INCLUDE).stream() + .flatMap(param -> Arrays.stream(param.split(","))) + .map(pathString -> new Path(entityClass, dictionary, pathString)) + .collect(Collectors.toSet()); + } + + return new HashSet<>(); + } + + private Set toRelationshipSet(Map relationships) { + return relationships.entrySet().stream() + .map(entry -> Relationship.builder() + .name(entry.getKey()) + .alias(entry.getKey()) + .projection(entry.getValue()) + .build()) + .collect(Collectors.toSet()); + } +} diff --git a/elide-core/src/main/java/com/yahoo/elide/jsonapi/document/processors/IncludedProcessor.java b/elide-core/src/main/java/com/yahoo/elide/jsonapi/document/processors/IncludedProcessor.java index 40056b0c11..a7805da4bb 100644 --- a/elide-core/src/main/java/com/yahoo/elide/jsonapi/document/processors/IncludedProcessor.java +++ b/elide-core/src/main/java/com/yahoo/elide/jsonapi/document/processors/IncludedProcessor.java @@ -7,8 +7,10 @@ import com.yahoo.elide.core.PersistentResource; import com.yahoo.elide.core.exceptions.ForbiddenAccessException; -import com.yahoo.elide.core.filter.expression.FilterExpression; +import com.yahoo.elide.jsonapi.EntityProjectionMaker; import com.yahoo.elide.jsonapi.models.JsonApiDocument; +import com.yahoo.elide.request.EntityProjection; +import com.yahoo.elide.request.Relationship; import com.google.common.collect.Lists; @@ -60,13 +62,16 @@ public void execute(JsonApiDocument jsonApiDocument, Set res */ private void addIncludedResources(JsonApiDocument jsonApiDocument, PersistentResource rec, List requestedRelationPaths) { + + EntityProjectionMaker maker = new EntityProjectionMaker(rec.getDictionary(), rec.getRequestScope()); + EntityProjection projection = maker.parseInclude(rec.getResourceClass()); // Process each include relation path requestedRelationPaths.forEach(pathParam -> { List pathList = Arrays.asList(pathParam.split(RELATION_PATH_SEPARATOR)); pathList.forEach(requestedRelationPath -> { List relationPath = Lists.newArrayList(requestedRelationPath.split(RELATION_PATH_DELIMITER)); - addResourcesForPath(jsonApiDocument, rec, relationPath); + addResourcesForPath(jsonApiDocument, rec, relationPath, projection); }); }); } @@ -76,15 +81,17 @@ private void addIncludedResources(JsonApiDocument jsonApiDocument, PersistentRes * JsonApiDocument. */ private void addResourcesForPath(JsonApiDocument jsonApiDocument, PersistentResource rec, - List relationPath) { + List relationPath, + EntityProjection projection) { //Pop off a relation of relation path String relation = relationPath.remove(0); - Optional filterExpression = rec.getRequestScope().getExpressionForRelation(rec, relation); Set collection; + Relationship relationship = projection.getRelationship(relation).orElseThrow(IllegalStateException::new); try { - collection = rec.getRelationCheckedFiltered(relation, filterExpression, Optional.empty(), Optional.empty()); + collection = rec.getRelationCheckedFiltered(relationship); + } catch (ForbiddenAccessException e) { return; } @@ -95,7 +102,8 @@ private void addResourcesForPath(JsonApiDocument jsonApiDocument, PersistentReso //If more relations left in the path, process a level deeper if (!relationPath.isEmpty()) { //Use a copy of the relationPath to preserve the path for remaining branches of the relationship tree - addResourcesForPath(jsonApiDocument, resource, new ArrayList<>(relationPath)); + addResourcesForPath(jsonApiDocument, resource, new ArrayList<>(relationPath), + relationship.getProjection()); } }); } diff --git a/elide-core/src/main/java/com/yahoo/elide/jsonapi/models/Resource.java b/elide-core/src/main/java/com/yahoo/elide/jsonapi/models/Resource.java index 9b71f59c6d..ac6cd3388a 100644 --- a/elide-core/src/main/java/com/yahoo/elide/jsonapi/models/Resource.java +++ b/elide-core/src/main/java/com/yahoo/elide/jsonapi/models/Resource.java @@ -5,12 +5,14 @@ */ package com.yahoo.elide.jsonapi.models; +import com.yahoo.elide.core.EntityDictionary; import com.yahoo.elide.core.PersistentResource; import com.yahoo.elide.core.RequestScope; import com.yahoo.elide.core.exceptions.ForbiddenAccessException; import com.yahoo.elide.core.exceptions.InvalidObjectIdentifierException; import com.yahoo.elide.core.exceptions.UnknownEntityException; import com.yahoo.elide.jsonapi.serialization.KeySerializer; +import com.yahoo.elide.request.EntityProjection; import com.fasterxml.jackson.annotation.JsonInclude; import com.fasterxml.jackson.annotation.JsonProperty; @@ -147,13 +149,20 @@ public boolean equals(Object obj) { public PersistentResource toPersistentResource(RequestScope requestScope) throws ForbiddenAccessException, InvalidObjectIdentifierException { - Class cls = requestScope.getDictionary().getEntityClass(type); + EntityDictionary dictionary = requestScope.getDictionary(); + Class cls = dictionary.getEntityClass(type); + if (cls == null) { throw new UnknownEntityException(type); } if (id == null) { throw new InvalidObjectIdentifierException(id, type); } - return PersistentResource.loadRecord(cls, id, requestScope); + + EntityProjection projection = EntityProjection.builder() + .type(cls) + .build(); + + return PersistentResource.loadRecord(projection, id, requestScope); } } diff --git a/elide-core/src/main/java/com/yahoo/elide/jsonapi/models/ResourceIdentifier.java b/elide-core/src/main/java/com/yahoo/elide/jsonapi/models/ResourceIdentifier.java index 5cf97420b5..726178e883 100644 --- a/elide-core/src/main/java/com/yahoo/elide/jsonapi/models/ResourceIdentifier.java +++ b/elide-core/src/main/java/com/yahoo/elide/jsonapi/models/ResourceIdentifier.java @@ -9,6 +9,7 @@ import com.yahoo.elide.core.RequestScope; import com.yahoo.elide.core.exceptions.ForbiddenAccessException; import com.yahoo.elide.core.exceptions.InvalidObjectIdentifierException; +import com.yahoo.elide.request.EntityProjection; import com.fasterxml.jackson.annotation.JsonProperty; @@ -38,7 +39,9 @@ public String getId() { public PersistentResource toPersistentResource(RequestScope requestScope) throws ForbiddenAccessException, InvalidObjectIdentifierException { Class cls = requestScope.getDictionary().getEntityClass(type); - return PersistentResource.loadRecord(cls, id, requestScope); + return PersistentResource.loadRecord(EntityProjection.builder() + .type(cls) + .build(), id, requestScope); } public Resource castToResource() { diff --git a/elide-core/src/main/java/com/yahoo/elide/parsers/state/CollectionTerminalState.java b/elide-core/src/main/java/com/yahoo/elide/parsers/state/CollectionTerminalState.java index 7673900c80..7ca47bdd33 100644 --- a/elide-core/src/main/java/com/yahoo/elide/parsers/state/CollectionTerminalState.java +++ b/elide-core/src/main/java/com/yahoo/elide/parsers/state/CollectionTerminalState.java @@ -14,9 +14,7 @@ import com.yahoo.elide.core.exceptions.InvalidObjectIdentifierException; import com.yahoo.elide.core.exceptions.InvalidValueException; import com.yahoo.elide.core.exceptions.UnknownEntityException; -import com.yahoo.elide.core.filter.expression.FilterExpression; import com.yahoo.elide.core.pagination.Pagination; -import com.yahoo.elide.core.sort.Sorting; import com.yahoo.elide.jsonapi.JsonApiMapper; import com.yahoo.elide.jsonapi.document.processors.DocumentProcessor; import com.yahoo.elide.jsonapi.document.processors.IncludedProcessor; @@ -25,11 +23,11 @@ import com.yahoo.elide.jsonapi.models.Meta; import com.yahoo.elide.jsonapi.models.Relationship; import com.yahoo.elide.jsonapi.models.Resource; +import com.yahoo.elide.request.EntityProjection; import com.fasterxml.jackson.core.JsonProcessingException; import com.fasterxml.jackson.databind.JsonNode; import com.google.common.base.Preconditions; - import org.apache.commons.collections4.IterableUtils; import org.apache.commons.lang3.tuple.Pair; @@ -56,9 +54,11 @@ public class CollectionTerminalState extends BaseState { private final Optional relationName; private final Class entityClass; private PersistentResource newObject; + private final EntityProjection parentProjection; public CollectionTerminalState(Class entityClass, Optional parent, - Optional relationName) { + Optional relationName, EntityProjection projection) { + this.parentProjection = projection; this.parent = parent; this.relationName = relationName; this.entityClass = entityClass; @@ -126,27 +126,13 @@ private Set getResourceCollection(RequestScope requestScope) // TODO: In case of join filters, apply pagination after getting records // instead of passing it to the datastore - Optional pagination = Optional.ofNullable(requestScope.getPagination()); - Optional sorting = Optional.ofNullable(requestScope.getSorting()); - if (parent.isPresent()) { - Optional filterExpression = - requestScope.getExpressionForRelation(parent.get(), relationName.get()); - collection = parent.get().getRelationCheckedFiltered( - relationName.get(), - filterExpression, - sorting, - pagination); + parentProjection.getRelationship(relationName.get()).orElseThrow(IllegalStateException::new)); } else { - Optional filterExpression = requestScope.getLoadFilterExpression(entityClass); - collection = PersistentResource.loadRecords( - entityClass, + parentProjection, new ArrayList<>(), //Empty list of IDs - filterExpression, - sorting, - pagination, requestScope); } @@ -187,8 +173,8 @@ private PersistentResource createObject(RequestScope requestScope) + " to type: " + entityClass); } - PersistentResource pResource = PersistentResource.createObject( - parent.orElse(null), newObjectClass, requestScope, Optional.ofNullable(id)); + PersistentResource pResource = PersistentResource.createObject(parent.orElse(null), newObjectClass, + requestScope, Optional.ofNullable(id)); Map attributes = resource.getAttributes(); if (attributes != null) { diff --git a/elide-core/src/main/java/com/yahoo/elide/parsers/state/RecordState.java b/elide-core/src/main/java/com/yahoo/elide/parsers/state/RecordState.java index f1c759fc22..a49ffcbb23 100644 --- a/elide-core/src/main/java/com/yahoo/elide/parsers/state/RecordState.java +++ b/elide-core/src/main/java/com/yahoo/elide/parsers/state/RecordState.java @@ -8,14 +8,13 @@ import com.yahoo.elide.core.EntityDictionary; import com.yahoo.elide.core.PersistentResource; import com.yahoo.elide.core.RelationshipType; -import com.yahoo.elide.core.exceptions.InvalidAttributeException; -import com.yahoo.elide.core.exceptions.InvalidCollectionException; -import com.yahoo.elide.core.filter.expression.FilterExpression; import com.yahoo.elide.generated.parsers.CoreParser.SubCollectionReadCollectionContext; import com.yahoo.elide.generated.parsers.CoreParser.SubCollectionReadEntityContext; import com.yahoo.elide.generated.parsers.CoreParser.SubCollectionRelationshipContext; import com.yahoo.elide.generated.parsers.CoreParser.SubCollectionSubCollectionContext; import com.yahoo.elide.jsonapi.models.SingleElementSet; +import com.yahoo.elide.request.EntityProjection; +import com.yahoo.elide.request.Relationship; import com.google.common.base.Preconditions; @@ -28,54 +27,52 @@ public class RecordState extends BaseState { private final PersistentResource resource; - public RecordState(PersistentResource resource) { + /* The projection which loaded this record */ + private final EntityProjection projection; + + public RecordState(PersistentResource resource, EntityProjection projection) { Preconditions.checkNotNull(resource); this.resource = resource; + this.projection = projection; } @Override public void handle(StateContext state, SubCollectionReadCollectionContext ctx) { String subCollection = ctx.term().getText(); EntityDictionary dictionary = state.getRequestScope().getDictionary(); + Class entityClass; String entityName; - try { - RelationshipType type = dictionary.getRelationshipType(resource.getObject(), subCollection); - if (type == RelationshipType.NONE) { - throw new InvalidCollectionException(subCollection); - } - Class paramType = dictionary.getParameterizedType(resource.getObject(), subCollection); - if (dictionary.isMappedInterface(paramType)) { - entityName = EntityDictionary.getSimpleName(paramType); - entityClass = paramType; - } else { - entityName = dictionary.getJsonAliasFor(paramType); - entityClass = dictionary.getEntityClass(entityName); - - } - if (entityClass == null) { - throw new IllegalArgumentException("Unknown type " + entityName); - } - final BaseState nextState; - final CollectionTerminalState collectionTerminalState = - new CollectionTerminalState(entityClass, Optional.of(resource), Optional.of(subCollection)); - Set collection = null; - if (type.isToOne()) { - Optional filterExpression = - state.getRequestScope().getExpressionForRelation(resource, subCollection); - collection = resource.getRelationCheckedFiltered(subCollection, - filterExpression, Optional.empty(), Optional.empty()); - } - if (collection instanceof SingleElementSet) { - PersistentResource record = ((SingleElementSet) collection).getValue(); - nextState = new RecordTerminalState(record, collectionTerminalState); - } else { - nextState = collectionTerminalState; - } - state.setState(nextState); - } catch (InvalidAttributeException e) { - throw new InvalidCollectionException(subCollection); + + RelationshipType type = dictionary.getRelationshipType(resource.getObject(), subCollection); + + Class paramType = dictionary.getParameterizedType(resource.getObject(), subCollection); + if (dictionary.isMappedInterface(paramType)) { + entityName = EntityDictionary.getSimpleName(paramType); + entityClass = paramType; + } else { + entityName = dictionary.getJsonAliasFor(paramType); + entityClass = dictionary.getEntityClass(entityName); } + if (entityClass == null) { + throw new IllegalArgumentException("Unknown type " + entityName); + } + final BaseState nextState; + final CollectionTerminalState collectionTerminalState = + new CollectionTerminalState(entityClass, Optional.of(resource), + Optional.of(subCollection), projection); + Set collection = null; + if (type.isToOne()) { + collection = resource.getRelationCheckedFiltered(projection.getRelationship(subCollection) + .orElseThrow(IllegalStateException::new)); + } + if (collection instanceof SingleElementSet) { + PersistentResource record = ((SingleElementSet) collection).getValue(); + nextState = new RecordTerminalState(record, collectionTerminalState); + } else { + nextState = collectionTerminalState; + } + state.setState(nextState); } @Override @@ -83,46 +80,35 @@ public void handle(StateContext state, SubCollectionReadEntityContext ctx) { String id = ctx.entity().id().getText(); String subCollection = ctx.entity().term().getText(); - try { - PersistentResource nextRecord = resource.getRelation(subCollection, id); - state.setState(new RecordTerminalState(nextRecord)); - } catch (InvalidAttributeException e) { - throw new InvalidCollectionException(subCollection); - } + PersistentResource nextRecord = resource.getRelation( + projection.getRelationship(subCollection).orElseThrow(IllegalStateException::new), id); + state.setState(new RecordTerminalState(nextRecord)); } @Override public void handle(StateContext state, SubCollectionSubCollectionContext ctx) { String id = ctx.entity().id().getText(); String subCollection = ctx.entity().term().getText(); - try { - state.setState(new RecordState(resource.getRelation(subCollection, id))); - } catch (InvalidAttributeException e) { - throw new InvalidCollectionException(subCollection); - } + + Relationship relationship = projection.getRelationship(subCollection) + .orElseThrow(IllegalStateException::new); + + state.setState(new RecordState(resource.getRelation(relationship, id), relationship.getProjection())); } @Override public void handle(StateContext state, SubCollectionRelationshipContext ctx) { String id = ctx.entity().id().getText(); String subCollection = ctx.entity().term().getText(); + String relationName = ctx.relationship().term().getText(); PersistentResource childRecord; - try { - childRecord = resource.getRelation(subCollection, id); - } catch (InvalidAttributeException e) { - throw new InvalidCollectionException(subCollection); - } - String relationName = ctx.relationship().term().getText(); - try { - Optional filterExpression = - state.getRequestScope().getExpressionForRelation(resource, subCollection); - childRecord.getRelationCheckedFiltered(relationName, filterExpression, Optional.empty(), Optional.empty()); - } catch (InvalidAttributeException e) { - throw new InvalidCollectionException(relationName); - } + Relationship childRelationship = projection.getRelationship(subCollection) + .orElseThrow(IllegalStateException::new); + + childRecord = resource.getRelation(childRelationship , id); - state.setState(new RelationshipTerminalState(childRecord, relationName)); + state.setState(new RelationshipTerminalState(childRecord, relationName, childRelationship.getProjection())); } } diff --git a/elide-core/src/main/java/com/yahoo/elide/parsers/state/RelationshipTerminalState.java b/elide-core/src/main/java/com/yahoo/elide/parsers/state/RelationshipTerminalState.java index 0c0ed93346..ae096090a0 100644 --- a/elide-core/src/main/java/com/yahoo/elide/parsers/state/RelationshipTerminalState.java +++ b/elide-core/src/main/java/com/yahoo/elide/parsers/state/RelationshipTerminalState.java @@ -18,6 +18,7 @@ import com.yahoo.elide.jsonapi.models.JsonApiDocument; import com.yahoo.elide.jsonapi.models.Relationship; import com.yahoo.elide.jsonapi.models.Resource; +import com.yahoo.elide.request.EntityProjection; import com.fasterxml.jackson.databind.JsonNode; @@ -41,8 +42,13 @@ public class RelationshipTerminalState extends BaseState { private final RelationshipType relationshipType; private final String relationshipName; - public RelationshipTerminalState(PersistentResource record, String relationshipName) { + /* The projection which loaded the resource which owns the relationship */ + private final EntityProjection parentProjection; + + public RelationshipTerminalState(PersistentResource record, String relationshipName, + EntityProjection parentProjection) { this.record = record; + this.parentProjection = parentProjection; this.relationshipType = record.getRelationshipType(relationshipName); this.relationshipName = relationshipName; @@ -55,7 +61,8 @@ public Supplier> handleGet(StateContext state) { JsonApiMapper mapper = requestScope.getMapper(); Optional> queryParams = requestScope.getQueryParams(); - Map relationships = record.toResourceWithSortingAndPagination().getRelationships(); + Map relationships = record.toResource(parentProjection).getRelationships(); + Relationship relationship = null; if (relationships != null) { Relationship relationship = relationships.get(relationshipName); diff --git a/elide-core/src/main/java/com/yahoo/elide/parsers/state/StartState.java b/elide-core/src/main/java/com/yahoo/elide/parsers/state/StartState.java index dfdba1827c..7b6734a85b 100644 --- a/elide-core/src/main/java/com/yahoo/elide/parsers/state/StartState.java +++ b/elide-core/src/main/java/com/yahoo/elide/parsers/state/StartState.java @@ -7,14 +7,12 @@ import com.yahoo.elide.core.EntityDictionary; import com.yahoo.elide.core.PersistentResource; -import com.yahoo.elide.core.exceptions.InvalidAttributeException; -import com.yahoo.elide.core.exceptions.InvalidCollectionException; -import com.yahoo.elide.core.filter.expression.FilterExpression; import com.yahoo.elide.generated.parsers.CoreParser.EntityContext; import com.yahoo.elide.generated.parsers.CoreParser.RootCollectionLoadEntitiesContext; import com.yahoo.elide.generated.parsers.CoreParser.RootCollectionLoadEntityContext; import com.yahoo.elide.generated.parsers.CoreParser.RootCollectionRelationshipContext; import com.yahoo.elide.generated.parsers.CoreParser.RootCollectionSubCollectionContext; +import com.yahoo.elide.request.EntityProjection; import java.util.Optional; @@ -27,10 +25,9 @@ public void handle(StateContext state, RootCollectionLoadEntitiesContext ctx) { String entityName = ctx.term().getText(); EntityDictionary dictionary = state.getRequestScope().getDictionary(); Class entityClass = dictionary.getEntityClass(entityName); - if (entityClass == null || !dictionary.isRoot(entityClass)) { - throw new InvalidCollectionException(entityName); - } - state.setState(new CollectionTerminalState(entityClass, Optional.empty(), Optional.empty())); + + state.setState(new CollectionTerminalState(entityClass, Optional.empty(), Optional.empty(), + state.getRequestScope().getEntityProjection())); } @Override @@ -42,23 +39,21 @@ public void handle(StateContext state, RootCollectionLoadEntityContext ctx) { @Override public void handle(StateContext state, RootCollectionSubCollectionContext ctx) { PersistentResource record = entityRecord(state, ctx.entity()); - state.setState(new RecordState(record)); + + state.setState(new RecordState(record, state.getRequestScope().getEntityProjection())); } @Override public void handle(StateContext state, RootCollectionRelationshipContext ctx) { PersistentResource record = entityRecord(state, ctx.entity()); + EntityProjection projection = state.getRequestScope().getEntityProjection(); String relationName = ctx.relationship().term().getText(); - try { - Optional filterExpression = - state.getRequestScope().getExpressionForRelation(record, relationName); - record.getRelationCheckedFiltered(relationName, filterExpression, Optional.empty(), Optional.empty()); - } catch (InvalidAttributeException e) { - throw new InvalidCollectionException(relationName); - } - state.setState(new RelationshipTerminalState(record, relationName)); + record.getRelationCheckedFiltered(projection.getRelationship(relationName) + .orElseThrow(IllegalStateException::new)); + + state.setState(new RelationshipTerminalState(record, relationName, projection)); } @Override @@ -67,14 +62,9 @@ public String toString() { } private PersistentResource entityRecord(StateContext state, EntityContext entity) { - String entityName = entity.term().getText(); String id = entity.id().getText(); - EntityDictionary dictionary = state.getRequestScope().getDictionary(); - Class entityClass = dictionary.getEntityClass(entityName); - if (entityClass == null || !dictionary.isRoot(entityClass)) { - throw new InvalidCollectionException(entityName); - } - return PersistentResource.loadRecord(entityClass, id, state.getRequestScope()); + return PersistentResource.loadRecord(state.getRequestScope().getEntityProjection(), + id, state.getRequestScope()); } } diff --git a/elide-core/src/main/java/com/yahoo/elide/request/Argument.java b/elide-core/src/main/java/com/yahoo/elide/request/Argument.java new file mode 100644 index 0000000000..9cc32089ba --- /dev/null +++ b/elide-core/src/main/java/com/yahoo/elide/request/Argument.java @@ -0,0 +1,32 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ + +package com.yahoo.elide.request; + +import lombok.Builder; +import lombok.Data; +import lombok.NonNull; + +/** + * Represents an argument passed to an attribute. + */ +@Data +@Builder +public class Argument { + + @NonNull + String name; + + Object value; + + /** + * Returns the argument type. + * @return the argument type. + */ + public Class getType() { + return value.getClass(); + } +} diff --git a/elide-core/src/main/java/com/yahoo/elide/request/Attribute.java b/elide-core/src/main/java/com/yahoo/elide/request/Attribute.java new file mode 100644 index 0000000000..7a05e71fd4 --- /dev/null +++ b/elide-core/src/main/java/com/yahoo/elide/request/Attribute.java @@ -0,0 +1,43 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ + +package com.yahoo.elide.request; + +import lombok.Builder; +import lombok.Data; +import lombok.NonNull; +import lombok.Singular; +import lombok.ToString; + +import java.util.Set; + +/** + * Represents an attribute on an Elide entity. Attributes can take arguments. + */ +@Data +@Builder +public class Attribute { + @NonNull + @ToString.Exclude + private Class type; + + @NonNull + private String name; + + @ToString.Exclude + private String alias; + + @Singular + @ToString.Exclude + private Set arguments; + + private Attribute(@NonNull Class type, @NonNull String name, String alias, Set arguments) { + this.type = type; + this.name = name; + this.alias = alias == null ? name : alias; + this.arguments = arguments; + } +} diff --git a/elide-core/src/main/java/com/yahoo/elide/request/EntityProjection.java b/elide-core/src/main/java/com/yahoo/elide/request/EntityProjection.java new file mode 100644 index 0000000000..da7a5c8caa --- /dev/null +++ b/elide-core/src/main/java/com/yahoo/elide/request/EntityProjection.java @@ -0,0 +1,261 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ + +package com.yahoo.elide.request; + +import com.yahoo.elide.core.filter.expression.FilterExpression; +import com.yahoo.elide.core.pagination.Pagination; +import com.yahoo.elide.core.sort.Sorting; + +import com.google.common.collect.Sets; +import lombok.AllArgsConstructor; +import lombok.Builder; +import lombok.Data; +import lombok.Getter; +import lombok.NonNull; + +import java.util.LinkedHashSet; +import java.util.Optional; +import java.util.Set; + +import javax.ws.rs.BadRequestException; + +/** + * Represents a client data request against a subgraph of the entity relationship graph. + */ +@Data +@Builder +@AllArgsConstructor +public class EntityProjection { + @NonNull + private Class type; + + private Set attributes; + + private Set relationships; + + private FilterExpression filterExpression; + + private Sorting sorting; + + private Pagination pagination; + + /** + * Creates a builder initialized as a copy of this collection + * @return The new builder + */ + public EntityProjectionBuilder copyOf() { + return EntityProjection.builder() + .type(this.type) + .attributes(new LinkedHashSet<>(attributes)) + .relationships(new LinkedHashSet<>(this.relationships)) + .filterExpression(this.filterExpression) + .sorting(this.sorting) + .pagination(this.pagination); + } + + /** + * Returns a relationship subgraph by name. + * @param name The name of the relationship. + * @return + */ + public Optional getRelationship(String name) { + return relationships.stream() + .filter((relationship) -> relationship.getName().equalsIgnoreCase(name)) + .findFirst(); + } + + /** + * Returns a relationship subgraph by name. + * @param name The name of the relationship. + * @param name The alias of the relationship. + * @return + */ + public Optional getRelationship(String name, String alias) { + return relationships.stream() + .filter((relationship) -> relationship.getName().equalsIgnoreCase(name)) + .filter((relationship) -> relationship.getAlias().equalsIgnoreCase(alias)) + .findFirst(); + } + + /** + * Recursively merges two EntityProjections. + * @param toMerge The projection to merge + * @return A newly created and merged EntityProjection. + */ + public EntityProjection merge(EntityProjection toMerge) { + EntityProjectionBuilder merged = copyOf(); + + for (Relationship relationship: toMerge.getRelationships()) { + EntityProjection theirs = relationship.getProjection(); + + Relationship ourRelationship = getRelationship(relationship.getName(), + relationship.getAlias()).orElse(null); + + if (ourRelationship != null) { + merged.relationships.remove(ourRelationship); + merged.relationships.add((Relationship.builder() + .name(relationship.getName()) + .alias(relationship.getAlias()) + .projection(ourRelationship.getProjection().merge(theirs)) + .build())); + } else { + merged.relationships.add((relationship)); + } + } + if (toMerge.getPagination() != null) { + merged.pagination = toMerge.getPagination(); + } + + if (toMerge.getSorting() != null) { + merged.sorting = toMerge.getSorting(); + } + + if (toMerge.getFilterExpression() != null) { + merged.filterExpression = toMerge.getFilterExpression(); + } + + merged.attributes.addAll(toMerge.attributes); + + return merged.build(); + } + + /** + * Customizes the lombok builder to our needs. + */ + public static class EntityProjectionBuilder { + @Getter + private Class type; + + private Set relationships = new LinkedHashSet<>(); + + private Set attributes = new LinkedHashSet<>(); + + @Getter + private FilterExpression filterExpression; + + @Getter + private Sorting sorting; + + @Getter + private Pagination pagination; + + public EntityProjectionBuilder relationships(Set relationships) { + this.relationships = relationships; + return this; + } + + public EntityProjectionBuilder attributes(Set attributes) { + this.attributes = attributes; + return this; + } + + public EntityProjectionBuilder relationship(String name, EntityProjection projection) { + return relationship(Relationship.builder() + .alias(name) + .name(name) + .projection(projection) + .build()); + } + + /** + * Add a new relationship into this project or merge an existing relationship that has same field name + * and alias as this relationship. If there exists another attribute/relationship of different field that is + * using the same alias, it would throw exception because that's ambiguous. + * + * @param relationship new relationship to add + * @return this builder after adding the relationship + */ + public EntityProjectionBuilder relationship(Relationship relationship) { + String relationshipName = relationship.getName(); + String relationshipAlias = relationship.getAlias(); + + Relationship existing = relationships.stream() + .filter(r -> r.getName().equals(relationshipName) && r.getAlias().equals(relationshipAlias)) + .findFirst().orElse(null); + + if (existing != null) { + relationships.remove(existing); + relationships.add(Relationship.builder() + .name(relationshipName) + .alias(relationshipAlias) + .projection(existing.getProjection().merge(relationship.getProjection())) + .build()); + } else { + if (isAmbiguous(relationshipName, relationshipAlias)) { + throw new BadRequestException( + String.format("Alias {%s}.{%s} is ambiguous.", type, relationshipAlias) + ); + } + relationships.add(relationship); + } + + return this; + } + + /** + * Add a new attribute into this project or merge an existing attribute that has same field name + * and alias as this attribute. If there exists another attribute/relationship of different field that is + * using the same alias, it would throw exception because that's ambiguous. + * + * @param attribute new attribute to add + * @return this builder after adding the attribute + */ + public EntityProjectionBuilder attribute(Attribute attribute) { + String attributeName = attribute.getName(); + String attributeAlias = attribute.getAlias(); + + Attribute existing = attributes.stream() + .filter(a -> a.getName().equals(attributeName) && a.getAlias().equals(attributeAlias)) + .findFirst().orElse(null); + + if (existing != null) { + attributes.remove(existing); + attributes.add(Attribute.builder() + .type(attribute.getType()) + .name(attributeName) + .alias(attributeAlias) + .arguments(Sets.union(attribute.getArguments(), existing.getArguments())) + .build()); + } else { + if (isAmbiguous(attributeName, attributeAlias)) { + throw new BadRequestException( + String.format("Alias {%s}.{%s} is ambiguous.", type, attributeAlias) + ); + } + attributes.add(attribute); + } + + return this; + } + + /** + * Get an attribute by alias. + * + * @param attributeAlias alias to refer to an attribute field + * @return found attribute or null + */ + public Attribute getAttributeByAlias(String attributeAlias) { + return attributes.stream() + .filter(attribute -> attribute.getAlias().equals(attributeAlias)) + .findAny() + .orElse(null); + } + + /** + * Check whether a field alias is ambiguous. + * + * @param fieldName field that the alias is bound to + * @param alias an field alias + * @return whether new alias would cause ambiguous + */ + private boolean isAmbiguous(String fieldName, String alias) { + return attributes.stream().anyMatch(a -> !fieldName.equals(a.getName()) && alias.equals(a.getAlias())) + || relationships.stream().anyMatch( + r -> !fieldName.equals(r.getName()) && alias.equals(r.getAlias())); + } + } +} diff --git a/elide-core/src/main/java/com/yahoo/elide/request/Relationship.java b/elide-core/src/main/java/com/yahoo/elide/request/Relationship.java new file mode 100644 index 0000000000..5cbbad0014 --- /dev/null +++ b/elide-core/src/main/java/com/yahoo/elide/request/Relationship.java @@ -0,0 +1,48 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ + +package com.yahoo.elide.request; + +import lombok.Builder; +import lombok.Data; +import lombok.NonNull; + +/** + * Represents a relationship on an Elide entity. + */ +@Data +@Builder +public class Relationship { + + public RelationshipBuilder copyOf() { + return Relationship.builder() + .alias(alias) + .name(name) + .projection(projection); + } + + @NonNull + private String name; + + private String alias; + + @NonNull + private EntityProjection projection; + + private Relationship(@NonNull String name, String alias, @NonNull EntityProjection projection) { + this.name = name; + this.alias = alias == null ? name : alias; + this.projection = projection; + } + + public Relationship merge(Relationship toMerge) { + return Relationship.builder() + .name(name) + .alias(alias) + .projection(projection.merge(toMerge.projection)) + .build(); + } +} diff --git a/elide-core/src/main/java/com/yahoo/elide/security/FilterExpressionCheck.java b/elide-core/src/main/java/com/yahoo/elide/security/FilterExpressionCheck.java index 95422dc9ea..6a03492201 100644 --- a/elide-core/src/main/java/com/yahoo/elide/security/FilterExpressionCheck.java +++ b/elide-core/src/main/java/com/yahoo/elide/security/FilterExpressionCheck.java @@ -18,6 +18,7 @@ import java.util.Optional; import java.util.function.Predicate; +import javax.inject.Inject; /** * Check for FilterExpression. This is a super class for user defined FilterExpression check. The subclass should @@ -28,6 +29,9 @@ @Slf4j public abstract class FilterExpressionCheck extends InlineCheck { + @Inject + protected EntityDictionary dictionary; + /** * Returns a FilterExpression from FilterExpressionCheck. * @@ -43,7 +47,6 @@ public final boolean ok(User user) { throw new UnsupportedOperationException(); } - /** * The filter expression is evaluated in memory if it cannot be pushed to the data store by elide for any reason. * @@ -54,7 +57,7 @@ public final boolean ok(User user) { */ @Override public final boolean ok(T object, RequestScope requestScope, Optional changeSpec) { - Class entityClass = coreScope(requestScope).getDictionary().lookupBoundClass(object.getClass()); + Class entityClass = dictionary.lookupBoundClass(object.getClass()); FilterExpression filterExpression = getFilterExpression(entityClass, requestScope); return filterExpression.accept(new FilterExpressionCheckEvaluationVisitor(object, this, requestScope)); } @@ -89,10 +92,23 @@ public boolean applyPredicateToObject(T object, FilterPredicate filterPredicate, * @param defaultPath path to use if no FieldExpressionPath defined * @return Predicates */ - protected static Path getFieldPath(Class type, RequestScope requestScope, String method, String defaultPath) { - EntityDictionary dictionary = coreScope(requestScope).getDictionary(); - FilterExpressionPath fep = dictionary.getMethodAnnotation(type, method, FilterExpressionPath.class); - return new Path(type, dictionary, fep == null ? defaultPath : fep.value()); + protected Path getFieldPath(Class type, RequestScope requestScope, String method, String defaultPath) { + try { + FilterExpressionPath fep = getFilterExpressionPath(type, method, dictionary); + return new Path(type, dictionary, fep == null ? defaultPath : fep.value()); + } catch (NoSuchMethodException | SecurityException e) { + throw new IllegalStateException(e); + } + } + + private static FilterExpressionPath getFilterExpressionPath( + Class type, + String method, + EntityDictionary dictionary) throws NoSuchMethodException { + FilterExpressionPath path = dictionary.lookupBoundClass(type) + .getMethod(method) + .getAnnotation(FilterExpressionPath.class); + return path; } protected static com.yahoo.elide.core.RequestScope coreScope(RequestScope requestScope) { diff --git a/elide-core/src/main/java/com/yahoo/elide/utils/ClassScanner.java b/elide-core/src/main/java/com/yahoo/elide/utils/ClassScanner.java index 767156a92b..7581eaad7c 100644 --- a/elide-core/src/main/java/com/yahoo/elide/utils/ClassScanner.java +++ b/elide-core/src/main/java/com/yahoo/elide/utils/ClassScanner.java @@ -10,6 +10,7 @@ import io.github.classgraph.ScanResult; import java.lang.annotation.Annotation; +import java.util.HashSet; import java.util.Set; import java.util.stream.Collectors; @@ -47,16 +48,19 @@ static public Set> getAnnotatedClasses(String packageName, Class> getAnnotatedClasses(Class annotation) { - try (ScanResult scanResult = new ClassGraph() - .enableClassInfo().enableAnnotationInfo().scan()) { - return scanResult.getClassesWithAnnotation(annotation.getCanonicalName()).stream() - .map((ClassInfo::loadClass)) - .collect(Collectors.toSet()); + static public Set> getAnnotatedClasses(Class ...annotations) { + Set> result = new HashSet<>(); + try (ScanResult scanResult = new ClassGraph().enableClassInfo().enableAnnotationInfo().scan()) { + for (Class annotation : annotations) { + result.addAll(scanResult.getClassesWithAnnotation(annotation.getCanonicalName()).stream() + .map((ClassInfo::loadClass)) + .collect(Collectors.toSet())); + } } + return result; } /** diff --git a/elide-core/src/test/java/com/yahoo/elide/core/EntityDictionaryTest.java b/elide-core/src/test/java/com/yahoo/elide/core/EntityDictionaryTest.java index fd419c5460..ecc1a1d6c3 100644 --- a/elide-core/src/test/java/com/yahoo/elide/core/EntityDictionaryTest.java +++ b/elide-core/src/test/java/com/yahoo/elide/core/EntityDictionaryTest.java @@ -24,9 +24,12 @@ import com.yahoo.elide.annotation.ReadPermission; import com.yahoo.elide.annotation.SecurityCheck; import com.yahoo.elide.core.exceptions.InvalidAttributeException; +import com.yahoo.elide.core.filter.expression.FilterExpression; import com.yahoo.elide.functions.LifeCycleHook; import com.yahoo.elide.models.generics.Employee; import com.yahoo.elide.models.generics.Manager; +import com.yahoo.elide.security.FilterExpressionCheck; +import com.yahoo.elide.security.RequestScope; import com.yahoo.elide.security.checks.UserCheck; import com.yahoo.elide.security.checks.prefab.Collections.AppendOnly; import com.yahoo.elide.security.checks.prefab.Collections.RemoveOnly; @@ -61,11 +64,12 @@ import java.util.Map; import java.util.Set; import java.util.stream.Collectors; - +import javax.inject.Inject; import javax.persistence.AccessType; import javax.persistence.Entity; import javax.persistence.GeneratedValue; import javax.persistence.Id; +import javax.persistence.OneToOne; import javax.persistence.Transient; public class EntityDictionaryTest extends EntityDictionary { @@ -106,22 +110,50 @@ public void testFindCheckByExpression() { assertEquals("Prefab.Common.UpdateOnCreate", getCheckIdentifier(UpdateOnCreate.class)); } - @SecurityCheck("User is Admin") - public class Foo extends UserCheck { + @Test + public void testCheckScan() { - @Override - public boolean ok(com.yahoo.elide.security.User user) { - return false; + @SecurityCheck("User is Admin") + class Bar extends UserCheck { + + @Override + public boolean ok(com.yahoo.elide.security.User user) { + return false; + } } + + EntityDictionary testDictionary = new EntityDictionary(new HashMap<>()); + testDictionary.scanForSecurityChecks(); + + assertEquals("User is Admin", testDictionary.getCheckIdentifier(Bar.class)); } + @Test - public void testCheckScan() { + public void testCheckInjection() { - EntityDictionary testDictionary = new EntityDictionary(new HashMap<>()); + @SecurityCheck("Filter Expression Injection Test") + class Foo extends FilterExpressionCheck { + + @Inject + Long testLong; + + @Override + public FilterExpression getFilterExpression(Class entityClass, RequestScope requestScope) { + assertEquals(testLong, 123L); + return null; + } + } + + EntityDictionary testDictionary = new EntityDictionary(new HashMap<>(), new Injector() { + @Override + public void inject(Object entity) { + ((Foo) entity).testLong = 123L; + } + }); testDictionary.scanForSecurityChecks(); - assertEquals("User is Admin", testDictionary.getCheckIdentifier(Foo.class)); + assertEquals("Filter Expression Injection Test", testDictionary.getCheckIdentifier(Foo.class)); } @Test @@ -528,7 +560,7 @@ class SubsubclassBinding extends SubclassBinding { bindEntity(SubclassBinding.class); bindEntity(SubsubclassBinding.class); - assertEquals(SubclassBinding.class, getEntityBinding(SubclassBinding.class).entityClass); + assertEquals(SuperclassBinding.class, getEntityBinding(SubclassBinding.class).entityClass); assertEquals(SuperclassBinding.class, getEntityBinding(SuperclassBinding.class).entityClass); assertEquals(SuperclassBinding.class, lookupEntityClass(SuperclassBinding.class)); @@ -565,11 +597,10 @@ class SubsubclassBinding extends SubclassBinding { } bindEntity(SuperclassBinding.class); - bindEntity(SubclassBinding.class); bindEntity(SubsubclassBinding.class); assertEquals(SuperclassBinding.class, getEntityBinding(SuperclassBinding.class).entityClass); - assertEquals(SubclassBinding.class, getEntityBinding(SubclassBinding.class).entityClass); + assertEquals(SuperclassBinding.class, getEntityBinding(SubclassBinding.class).entityClass); assertEquals(SubsubclassBinding.class, getEntityBinding(SubsubclassBinding.class).entityClass); assertEquals(SuperclassBinding.class, lookupEntityClass(SuperclassBinding.class)); @@ -606,15 +637,13 @@ class SubsubclassBinding extends SubclassBinding { } bindEntity(SuperclassBinding.class); - bindEntity(SubclassBinding.class); - bindEntity(SubsubclassBinding.class); - assertEquals(SubclassBinding.class, getEntityBinding(SubclassBinding.class).entityClass); + assertEquals(SuperclassBinding.class, getEntityBinding(SubclassBinding.class).entityClass); assertEquals(SuperclassBinding.class, getEntityBinding(SuperclassBinding.class).entityClass); assertEquals(SuperclassBinding.class, lookupIncludeClass(SuperclassBinding.class)); - assertEquals(SubclassBinding.class, lookupIncludeClass(SubclassBinding.class)); - assertEquals(SubsubclassBinding.class, lookupIncludeClass(SubsubclassBinding.class)); + assertEquals(SuperclassBinding.class, lookupIncludeClass(SubclassBinding.class)); + assertEquals(SuperclassBinding.class, lookupIncludeClass(SubsubclassBinding.class)); } @Test @@ -635,15 +664,14 @@ class SubsubclassBinding extends SubclassBinding { } bindEntity(SuperclassBinding.class); - bindEntity(SubclassBinding.class); bindEntity(SubsubclassBinding.class); - assertEquals(SubclassBinding.class, getEntityBinding(SubclassBinding.class).entityClass); + assertEquals(SuperclassBinding.class, getEntityBinding(SubclassBinding.class).entityClass); assertEquals(SuperclassBinding.class, getEntityBinding(SuperclassBinding.class).entityClass); assertEquals(SubsubclassBinding.class, getEntityBinding(SubsubclassBinding.class).entityClass); assertEquals(SuperclassBinding.class, lookupIncludeClass(SuperclassBinding.class)); - assertEquals(SubclassBinding.class, lookupIncludeClass(SubclassBinding.class)); + assertEquals(SuperclassBinding.class, lookupIncludeClass(SubclassBinding.class)); assertEquals(SubsubclassBinding.class, lookupIncludeClass(SubsubclassBinding.class)); } @@ -663,17 +691,16 @@ class SubsubclassBinding extends SubclassBinding { } bindEntity(SuperclassBinding.class); - bindEntity(SubclassBinding.class); bindEntity(SubsubclassBinding.class); - assertEquals(SubclassBinding.class, getEntityBinding(SubclassBinding.class).entityClass); + assertEquals(SuperclassBinding.class, getEntityBinding(SubclassBinding.class).entityClass); assertEquals(SuperclassBinding.class, getEntityBinding(SuperclassBinding.class).entityClass); assertThrows(IllegalArgumentException.class, () -> { getEntityBinding(SubsubclassBinding.class); }); assertEquals(SuperclassBinding.class, lookupIncludeClass(SuperclassBinding.class)); - assertEquals(SubclassBinding.class, lookupIncludeClass(SubclassBinding.class)); + assertEquals(SuperclassBinding.class, lookupIncludeClass(SubclassBinding.class)); assertEquals(null, lookupIncludeClass(SubsubclassBinding.class)); } @@ -831,4 +858,16 @@ public void testCheckLookup() throws Exception { assertThrows(IllegalArgumentException.class, () -> this.getCheck(String.class.getName())); } + + @Test + public void testAttributeOrRelationAnnotationExists() { + assertTrue(attributeOrRelationAnnotationExists(Job.class, "jobId", Id.class)); + assertFalse(attributeOrRelationAnnotationExists(Job.class, "title", OneToOne.class)); + } + + @Test + public void testIsValidField() { + assertTrue(isValidField(Job.class, "title")); + assertFalse(isValidField(Job.class, "foo")); + } } diff --git a/elide-core/src/test/java/com/yahoo/elide/core/LifeCycleTest.java b/elide-core/src/test/java/com/yahoo/elide/core/LifeCycleTest.java index 0e650a1add..a19a6f6f7e 100644 --- a/elide-core/src/test/java/com/yahoo/elide/core/LifeCycleTest.java +++ b/elide-core/src/test/java/com/yahoo/elide/core/LifeCycleTest.java @@ -46,6 +46,7 @@ import com.yahoo.elide.core.datastore.inmemory.HashMapDataStore; import com.yahoo.elide.core.datastore.inmemory.InMemoryDataStore; import com.yahoo.elide.functions.LifeCycleHook; +import com.yahoo.elide.request.EntityProjection; import com.yahoo.elide.security.ChangeSpec; import com.yahoo.elide.security.User; import com.yahoo.elide.security.checks.Check; @@ -58,7 +59,6 @@ import example.Book; import example.Editor; import example.Publisher; -import example.TestCheckMappings; import org.junit.jupiter.api.BeforeEach; import org.junit.jupiter.api.Test; @@ -66,7 +66,6 @@ import java.util.Arrays; import java.util.HashMap; import java.util.HashSet; -import java.util.Map; import java.util.Optional; import javax.persistence.Entity; @@ -89,7 +88,6 @@ public class LifeCycleTest { private MockCallback onUpdatePostCommitCallback; private MockCallback onUpdatePostCommitAuthor; - public class MockCallback implements LifeCycleHook { @Override public void execute(T object, com.yahoo.elide.security.RequestScope scope, Optional changes) { @@ -97,29 +95,13 @@ public void execute(T object, com.yahoo.elide.security.RequestScope scope, Optio } } - public class TestEntityDictionary extends EntityDictionary { - public TestEntityDictionary(Map> checks) { - super(checks); - } - - @Override - public Class lookupBoundClass(Class objClass) { - // Special handling for mocked Book class which has Entity annotation - if (objClass.getName().contains("$MockitoMock$")) { - objClass = objClass.getSuperclass(); - } - return super.lookupBoundClass(objClass); - } - - } - LifeCycleTest() throws Exception { callback = mock(MockCallback.class); onUpdateDeferredCallback = mock(MockCallback.class); onUpdateImmediateCallback = mock(MockCallback.class); onUpdatePostCommitCallback = mock(MockCallback.class); onUpdatePostCommitAuthor = mock(MockCallback.class); - dictionary = new TestEntityDictionary(TestCheckMappings.MAPPINGS); + dictionary = TestDictionary.getTestDictionary(); dictionary.bindEntity(Book.class); dictionary.bindEntity(Author.class); dictionary.bindEntity(Publisher.class); @@ -261,7 +243,7 @@ public void testElideGetRelationship() throws Exception { when(store.beginReadTransaction()).thenCallRealMethod(); when(store.beginTransaction()).thenReturn(tx); - when(tx.loadObject(eq(Book.class), any(), any(), isA(RequestScope.class))).thenReturn(book); + when(tx.loadObject(isA(EntityProjection.class), any(), isA(RequestScope.class))).thenReturn(book); MultivaluedMap headers = new MultivaluedHashMap<>(); ElideResponse response = elide.get("/book/1/relationships/authors", headers, null); @@ -291,7 +273,7 @@ public void testElidePatch() throws Exception { when(book.getId()).thenReturn(1L); when(store.beginTransaction()).thenReturn(tx); - when(tx.loadObject(eq(Book.class), any(), any(), isA(RequestScope.class))).thenReturn(book); + when(tx.loadObject(isA(EntityProjection.class), any(), isA(RequestScope.class))).thenReturn(book); String bookBody = "{\"data\":{\"type\":\"book\",\"id\":1,\"attributes\": {\"title\":\"Grapes of Wrath\"}}}"; @@ -378,7 +360,7 @@ public void testElideDelete() throws Exception { when(book.getId()).thenReturn(1L); when(store.beginTransaction()).thenReturn(tx); - when(tx.loadObject(eq(Book.class), any(), any(), isA(RequestScope.class))).thenReturn(book); + when(tx.loadObject(isA(EntityProjection.class), any(), isA(RequestScope.class))).thenReturn(book); ElideResponse response = elide.delete("/book/1", "", null); assertEquals(HttpStatus.SC_NO_CONTENT, response.getResponseCode()); @@ -405,7 +387,7 @@ public void testCreate() { DataStoreTransaction tx = mock(DataStoreTransaction.class); when(tx.createNewObject(Book.class)).thenReturn(book); RequestScope scope = buildRequestScope(dictionary, tx); - PersistentResource resource = PersistentResource.createObject(null, Book.class, scope, Optional.of("uuid")); + PersistentResource resource = PersistentResource.createObject(Book.class, scope, Optional.of("uuid")); resource.setValueChecked("title", "should not affect calls since this is create!"); resource.setValueChecked("genre", "boring books"); assertNotNull(resource); @@ -642,8 +624,8 @@ public void testUpdateRelationshipWithChangeSpec() { book.setAuthors(Sets.newHashSet(author)); author.setBooks(Sets.newHashSet(book)); DataStoreTransaction tx = mock(DataStoreTransaction.class); - when(tx.getRelation(any(), eq(author), eq("books"), any(), any(), any(), any())).then((i) -> author.getBooks()); - when(tx.getRelation(any(), eq(book), eq("authors"), any(), any(), any(), any())).then((i) -> book.getAuthors()); + when(tx.getRelation(any(), eq(author), any(), any())).then((i) -> author.getBooks()); + when(tx.getRelation(any(), eq(book), any(), any())).then((i) -> book.getAuthors()); RequestScope scope = buildRequestScope(dictionary, tx); PersistentResource resourceBook = new PersistentResource(book, null, scope.getUUIDFor(book), scope); @@ -766,7 +748,7 @@ public void blowUp(RequestScope scope) { } } - EntityDictionary dictionary = new EntityDictionary(new HashMap<>()); + EntityDictionary dictionary = TestDictionary.getTestDictionary(); DataStoreTransaction tx = mock(DataStoreTransaction.class); dictionary.bindEntity(Book.class); @@ -790,7 +772,7 @@ public void blowUp(RequestScope scope) { } } - EntityDictionary dictionary = new EntityDictionary(new HashMap<>()); + EntityDictionary dictionary = TestDictionary.getTestDictionary(); DataStoreTransaction tx = mock(DataStoreTransaction.class); dictionary.bindEntity(Book.class); @@ -844,7 +826,7 @@ public void readPostCommit(RequestScope scope) { } } - EntityDictionary dictionary = new EntityDictionary(new HashMap<>()); + EntityDictionary dictionary = TestDictionary.getTestDictionary(); DataStoreTransaction tx = mock(DataStoreTransaction.class); dictionary.bindEntity(Book.class); @@ -906,7 +888,7 @@ public void updatePostCommit(RequestScope scope) { } } - EntityDictionary dictionary = new EntityDictionary(new HashMap<>()); + EntityDictionary dictionary = TestDictionary.getTestDictionary(); DataStoreTransaction tx = mock(DataStoreTransaction.class); dictionary.bindEntity(Book.class); @@ -968,7 +950,7 @@ public void createPostCommit(RequestScope scope) { } } - EntityDictionary dictionary = new EntityDictionary(new HashMap<>()); + EntityDictionary dictionary = TestDictionary.getTestDictionary(); DataStoreTransaction tx = mock(DataStoreTransaction.class); dictionary.bindEntity(Book.class); @@ -976,6 +958,7 @@ public void createPostCommit(RequestScope scope) { when(tx.createNewObject(Book.class)).thenReturn(book); RequestScope scope = buildRequestScope(dictionary, tx); PersistentResource bookResource = PersistentResource.createObject(null, Book.class, scope, Optional.of("123")); + PersistentResource bookResource = PersistentResource.createObject(Book.class, scope, Optional.of("123")); bookResource.updateAttribute("title", "Foo"); assertEquals(0, book.createPreSecurityInvoked); @@ -1032,7 +1015,7 @@ public void deletePostCommit(RequestScope scope) { } } - EntityDictionary dictionary = new EntityDictionary(new HashMap<>()); + EntityDictionary dictionary = TestDictionary.getTestDictionary(); DataStoreTransaction tx = mock(DataStoreTransaction.class); dictionary.bindEntity(Book.class); @@ -1064,12 +1047,16 @@ public void testAddToCollectionTrigger() { HashMap> checkMappings = new HashMap<>(); checkMappings.put("Book operation check", Book.BookOperationCheck.class); checkMappings.put("Field path editor check", Editor.FieldPathFilterExpression.class); - store.populateEntityDictionary(new EntityDictionary(checkMappings)); + EntityDictionary dictionary = TestDictionary.getTestDictionary(checkMappings); + + store.populateEntityDictionary(dictionary); DataStoreTransaction tx = store.beginTransaction(); RequestScope scope = buildRequestScope(wrapped.getDictionary(), tx); - PersistentResource publisherResource = PersistentResource.createObject(null, Publisher.class, scope, Optional.of("1")); + + PersistentResource publisherResource = PersistentResource.createObject(Publisher.class, scope, Optional.of("1")); PersistentResource book1Resource = PersistentResource.createObject(publisherResource, Book.class, scope, Optional.of("1")); + publisherResource.updateRelation("books", new HashSet<>(Arrays.asList(book1Resource))); scope.runQueuedPreCommitTriggers(); @@ -1085,7 +1072,10 @@ public void testAddToCollectionTrigger() { scope = buildRequestScope(wrapped.getDictionary(), tx); PersistentResource book2Resource = PersistentResource.createObject(publisherResource, Book.class, scope, Optional.of("2")); - publisherResource = PersistentResource.loadRecord(Publisher.class, "1", scope); + publisherResource = PersistentResource.loadRecord( + EntityProjection.builder() + .type(Publisher.class) + .build(), "1", scope); publisherResource.addRelation("books", book2Resource); scope.runQueuedPreCommitTriggers(); @@ -1104,12 +1094,12 @@ public void testRemoveFromCollectionTrigger() { HashMap> checkMappings = new HashMap<>(); checkMappings.put("Book operation check", Book.BookOperationCheck.class); checkMappings.put("Field path editor check", Editor.FieldPathFilterExpression.class); - store.populateEntityDictionary(new EntityDictionary(checkMappings)); + store.populateEntityDictionary(TestDictionary.getTestDictionary(checkMappings)); DataStoreTransaction tx = store.beginTransaction(); RequestScope scope = buildRequestScope(wrapped.getDictionary(), tx); - PersistentResource publisherResource = PersistentResource.createObject(null, Publisher.class, scope, Optional.of("1")); + PersistentResource publisherResource = PersistentResource.createObject(Publisher.class, scope, Optional.of("1")); PersistentResource book1Resource = PersistentResource.createObject(publisherResource, Book.class, scope, Optional.of("1")); PersistentResource book2Resource = PersistentResource.createObject(publisherResource, Book.class, scope, Optional.of("2")); publisherResource.updateRelation("books", new HashSet<>(Arrays.asList(book1Resource, book2Resource))); @@ -1127,7 +1117,11 @@ public void testRemoveFromCollectionTrigger() { scope = buildRequestScope(wrapped.getDictionary(), tx); book2Resource = PersistentResource.createObject(publisherResource, Book.class, scope, Optional.of("2")); - publisherResource = PersistentResource.loadRecord(Publisher.class, "1", scope); + + publisherResource = PersistentResource.loadRecord(EntityProjection.builder() + .type(Publisher.class) + .build(), "1", scope); + publisherResource.updateRelation("books", new HashSet<>(Arrays.asList(book2Resource))); scope.runQueuedPreCommitTriggers(); diff --git a/elide-core/src/test/java/com/yahoo/elide/core/PermissionAnnotationTest.java b/elide-core/src/test/java/com/yahoo/elide/core/PermissionAnnotationTest.java index 080eebef53..8434bdea03 100644 --- a/elide-core/src/test/java/com/yahoo/elide/core/PermissionAnnotationTest.java +++ b/elide-core/src/test/java/com/yahoo/elide/core/PermissionAnnotationTest.java @@ -21,7 +21,6 @@ import com.yahoo.elide.security.executors.ActivePermissionExecutor; import example.FunWithPermissions; -import example.TestCheckMappings; import org.junit.jupiter.api.BeforeAll; import org.junit.jupiter.api.Test; @@ -35,7 +34,7 @@ public class PermissionAnnotationTest { private static PersistentResource funRecord; private static PersistentResource badRecord; - private static EntityDictionary dictionary = new EntityDictionary(TestCheckMappings.MAPPINGS); + private static EntityDictionary dictionary = TestDictionary.getTestDictionary(); public PermissionAnnotationTest() { } diff --git a/elide-core/src/test/java/com/yahoo/elide/core/PersistenceResourceTestSetup.java b/elide-core/src/test/java/com/yahoo/elide/core/PersistenceResourceTestSetup.java index 2692721f95..9558f6e917 100644 --- a/elide-core/src/test/java/com/yahoo/elide/core/PersistenceResourceTestSetup.java +++ b/elide-core/src/test/java/com/yahoo/elide/core/PersistenceResourceTestSetup.java @@ -42,7 +42,6 @@ import example.Parent; import example.Publisher; import example.Right; -import example.TestCheckMappings; import example.UpdateAndCreate; import example.packageshareable.ContainerWithPackageShare; import example.packageshareable.ShareableWithPackageShare; @@ -71,7 +70,8 @@ public class PersistenceResourceTestSetup extends PersistentResource { protected final ElideSettings elideSettings; protected static EntityDictionary initDictionary() { - EntityDictionary dictionary = new EntityDictionary(TestCheckMappings.MAPPINGS); + EntityDictionary dictionary = TestDictionary.getTestDictionary(); + dictionary.bindEntity(UpdateAndCreate.class); dictionary.bindEntity(Author.class); dictionary.bindEntity(Book.class); diff --git a/elide-core/src/test/java/com/yahoo/elide/core/PersistentResourceTest.java b/elide-core/src/test/java/com/yahoo/elide/core/PersistentResourceTest.java index 06744d79a7..8ab3be2bcc 100644 --- a/elide-core/src/test/java/com/yahoo/elide/core/PersistentResourceTest.java +++ b/elide-core/src/test/java/com/yahoo/elide/core/PersistentResourceTest.java @@ -38,6 +38,8 @@ import com.yahoo.elide.jsonapi.models.Relationship; import com.yahoo.elide.jsonapi.models.Resource; import com.yahoo.elide.jsonapi.models.ResourceIdentifier; +import com.yahoo.elide.request.Attribute; +import com.yahoo.elide.request.EntityProjection; import com.yahoo.elide.security.ChangeSpec; import com.yahoo.elide.security.User; @@ -67,12 +69,21 @@ import example.packageshareable.ContainerWithPackageShare; import example.packageshareable.ShareableWithPackageShare; import example.packageshareable.UnshareableWithEntityUnshare; +<<<<<<< HEAD import org.apache.commons.collections4.CollectionUtils; import org.apache.commons.collections4.IterableUtils; import org.junit.jupiter.api.Test; import org.mockito.Answers; +======= +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.Test; +import org.junit.jupiter.api.TestInstance; +import org.mockito.ArgumentCaptor; +import lombok.AllArgsConstructor; +import lombok.EqualsAndHashCode; +>>>>>>> ef111d6e... Create AggregationDataStore module (#845) import nocreate.NoCreateEntity; import java.util.ArrayList; @@ -92,19 +103,26 @@ import javax.ws.rs.core.MultivaluedHashMap; import javax.ws.rs.core.MultivaluedMap; - /** * Test PersistentResource. */ +@TestInstance(TestInstance.Lifecycle.PER_CLASS) public class PersistentResourceTest extends PersistenceResourceTestSetup { - private final RequestScope goodUserScope; - private final RequestScope badUserScope; + private final User goodUser = new User(1); + private final User badUser = new User(-1); + private DataStoreTransaction tx = mock(DataStoreTransaction.class); +<<<<<<< HEAD public PersistentResourceTest() { goodUserScope = buildRequestScope(mock(DataStoreTransaction.class), new User(1)); badUserScope = buildRequestScope(mock(DataStoreTransaction.class), new User(-1)); reset(goodUserScope.getTransaction()); +======= + @BeforeEach + public void beforeTest() { + reset(tx); +>>>>>>> ef111d6e... Create AggregationDataStore module (#845) } @Test @@ -112,11 +130,15 @@ public void testUpdateToOneRelationHookInAddRelation() { FunWithPermissions fun = new FunWithPermissions(); Child child = newChild(1); +<<<<<<< HEAD User goodUser = new User(1); DataStoreTransaction tx = mock(DataStoreTransaction.class); RequestScope goodScope = buildRequestScope(tx, goodUser); +======= + RequestScope goodScope = new RequestScope(null, null, tx, goodUser, null, elideSettings); +>>>>>>> ef111d6e... Create AggregationDataStore module (#845) PersistentResource funResource = new PersistentResource<>(fun, null, "3", goodScope); PersistentResource childResource = new PersistentResource<>(child, null, "1", goodScope); funResource.addRelation("relation3", childResource); @@ -131,9 +153,6 @@ public void testUpdateToOneRelationHookInUpdateRelation() { Child child1 = newChild(1); Child child2 = newChild(2); fun.setRelation1(Sets.newHashSet(child1)); - User goodUser = new User(1); - - DataStoreTransaction tx = mock(DataStoreTransaction.class); RequestScope goodScope = buildRequestScope(tx, goodUser); PersistentResource funResource = new PersistentResource<>(fun, null, "3", goodScope); @@ -150,9 +169,6 @@ public void testUpdateToOneRelationHookInRemoveRelation() { FunWithPermissions fun = new FunWithPermissions(); Child child = newChild(1); fun.setRelation3(child); - User goodUser = new User(1); - - DataStoreTransaction tx = mock(DataStoreTransaction.class); RequestScope goodScope = buildRequestScope(tx, goodUser); PersistentResource funResource = new PersistentResource<>(fun, null, "3", goodScope); @@ -168,12 +184,15 @@ public void testUpdateToOneRelationHookInClearRelation() { FunWithPermissions fun = new FunWithPermissions(); Child child1 = newChild(1); fun.setRelation3(child1); - User goodUser = new User(1); - DataStoreTransaction tx = mock(DataStoreTransaction.class); - when(tx.getRelation(any(), eq(fun), eq("relation3"), any(), any(), any(), any())).thenReturn(child1); + when(tx.getRelation(any(), eq(fun), any(), any())).thenReturn(child1); +<<<<<<< HEAD RequestScope goodScope = buildRequestScope(tx, goodUser); +======= + RequestScope goodScope = new RequestScope(null, null, tx, goodUser, null, elideSettings); + +>>>>>>> ef111d6e... Create AggregationDataStore module (#845) PersistentResource funResource = new PersistentResource<>(fun, null, "3", goodScope); funResource.clearRelation("relation3"); @@ -186,11 +205,16 @@ public void testUpdateToManyRelationHookInAddRelationBidirection() { Parent parent = new Parent(); Child child = newChild(1); +<<<<<<< HEAD User goodUser = new User(1); DataStoreTransaction tx = mock(DataStoreTransaction.class); RequestScope goodScope = buildRequestScope(tx, goodUser); +======= + RequestScope goodScope = new RequestScope(null, null, tx, goodUser, null, elideSettings); + +>>>>>>> ef111d6e... Create AggregationDataStore module (#845) PersistentResource parentResource = new PersistentResource<>(parent, null, "3", goodScope); PersistentResource childResource = new PersistentResource<>(child, null, "1", goodScope); parentResource.addRelation("children", childResource); @@ -206,9 +230,6 @@ public void testUpdateToManyRelationHookInRemoveRelationBidirection() { Child child = newChild(1); parent.setChildren(Sets.newHashSet(child)); child.setParents(Sets.newHashSet(parent)); - User goodUser = new User(1); - - DataStoreTransaction tx = mock(DataStoreTransaction.class); RequestScope goodScope = buildRequestScope(tx, goodUser); PersistentResource parentResource = new PersistentResource<>(parent, null, "3", goodScope); @@ -230,10 +251,8 @@ public void testUpdateToManyRelationHookInClearRelationBidirection() { parent.setChildren(children); child1.setParents(Sets.newHashSet(parent)); child2.setParents(Sets.newHashSet(parent)); - User goodUser = new User(1); - DataStoreTransaction tx = mock(DataStoreTransaction.class); - when(tx.getRelation(any(), eq(parent), eq("children"), any(), any(), any(), any())).thenReturn(children); + when(tx.getRelation(any(), eq(parent), any(), any())).thenReturn(children); RequestScope goodScope = buildRequestScope(tx, goodUser); PersistentResource parentResource = new PersistentResource<>(parent, null, "3", goodScope); @@ -257,10 +276,8 @@ public void testUpdateToManyRelationHookInUpdateRelationBidirection() { parent.setChildren(children); child1.setParents(Sets.newHashSet(parent)); child2.setParents(Sets.newHashSet(parent)); - User goodUser = new User(1); - DataStoreTransaction tx = mock(DataStoreTransaction.class); - when(tx.getRelation(any(), eq(parent), eq("children"), any(), any(), any(), any())).thenReturn(children); + when(tx.getRelation(any(), eq(parent), any(), any())).thenReturn(children); RequestScope goodScope = buildRequestScope(tx, goodUser); PersistentResource parentResource = new PersistentResource<>(parent, null, "3", goodScope); @@ -281,14 +298,16 @@ public void testUpdateToManyRelationHookInUpdateRelationBidirection() { @Test public void testSetAttributeHookInUpdateAttribute() { Parent parent = newParent(1); - DataStoreTransaction tx = mock(DataStoreTransaction.class); - User goodUser = new User(1); + ArgumentCaptor attributeArgument = ArgumentCaptor.forClass(Attribute.class); RequestScope goodScope = buildRequestScope(tx, goodUser); PersistentResource parentResource = new PersistentResource<>(parent, null, "1", goodScope); parentResource.updateAttribute("firstName", "foobar"); - verify(tx, times(1)).setAttribute(parent, "firstName", "foobar", goodScope); + verify(tx, times(1)).setAttribute(eq(parent), attributeArgument.capture(), eq(goodScope)); + + assertEquals(attributeArgument.getValue().getName(), "firstName"); + assertEquals(attributeArgument.getValue().getArguments().iterator().next().getValue(), "foobar"); } @Test @@ -298,7 +317,9 @@ public void testGetRelationships() { fun.setRelation2(Sets.newHashSet()); fun.setRelation3(null); - PersistentResource funResource = new PersistentResource<>(fun, null, "3", goodUserScope); + RequestScope scope = new TestRequestScope(tx, goodUser, dictionary); + + PersistentResource funResource = new PersistentResource<>(fun, null, "3", scope); Map relationships = funResource.getRelationships(); @@ -309,7 +330,9 @@ public void testGetRelationships() { assertTrue(relationships.containsKey("relation4"), "relation4 should be present"); assertTrue(relationships.containsKey("relation5"), "relation5 should be present"); - PersistentResource funResourceWithBadScope = new PersistentResource<>(fun, null, "3", badUserScope); + scope = new TestRequestScope(tx, badUser, dictionary); + + PersistentResource funResourceWithBadScope = new PersistentResource<>(fun, null, "3", scope); relationships = funResourceWithBadScope.getRelationships(); assertEquals(0, relationships.size(), "All relationships should be filtered out"); @@ -319,8 +342,6 @@ public void testGetRelationships() { public void testNoCreate() { assertNotNull(dictionary); NoCreateEntity noCreate = new NoCreateEntity(); - DataStoreTransaction tx = mock(DataStoreTransaction.class); - User goodUser = new User(1); when(tx.createNewObject(NoCreateEntity.class)).thenReturn(noCreate); @@ -328,7 +349,7 @@ public void testNoCreate() { assertThrows( ForbiddenAccessException.class, () -> PersistentResource.createObject( - null, NoCreateEntity.class, goodScope, Optional.of("1"))); // should throw here + NoCreateEntity.class, goodScope, Optional.of("1"))); // should throw here } @Test @@ -339,7 +360,8 @@ public void testGetAttributes() { fun.setField2(null); fun.setField4("bar"); - PersistentResource funResource = new PersistentResource<>(fun, null, "3", goodUserScope); + RequestScope scope = new TestRequestScope(tx, goodUser, dictionary); + PersistentResource funResource = new PersistentResource<>(fun, null, "3", scope); Map attributes = funResource.getAttributes(); @@ -357,6 +379,7 @@ public void testGetAttributes() { assertEquals(attributes.get("field3"), "Foobar", "field3 should be set to original value."); assertEquals(attributes.get("field4"), "bar", "field4 should be set to original value."); + RequestScope badUserScope = new TestRequestScope(tx, badUser, dictionary); PersistentResource funResourceBad = new PersistentResource<>(fun, null, "3", badUserScope); attributes = funResourceBad.getAttributes(); @@ -377,10 +400,11 @@ public void testFilter() { Child child4 = newChild(-4); { - PersistentResource child1Resource = new PersistentResource<>(child1, null, "1", goodUserScope); - PersistentResource child2Resource = new PersistentResource<>(child2, null, "-2", goodUserScope); - PersistentResource child3Resource = new PersistentResource<>(child3, null, "3", goodUserScope); - PersistentResource child4Resource = new PersistentResource<>(child4, null, "-4", goodUserScope); + RequestScope scope = new TestRequestScope(tx, goodUser, dictionary); + PersistentResource child1Resource = new PersistentResource<>(child1, null, "1", scope); + PersistentResource child2Resource = new PersistentResource<>(child2, null, "-2", scope); + PersistentResource child3Resource = new PersistentResource<>(child3, null, "3", scope); + PersistentResource child4Resource = new PersistentResource<>(child4, null, "-4", scope); Set resources = Sets.newHashSet(child1Resource, child2Resource, child3Resource, child4Resource); @@ -392,10 +416,11 @@ public void testFilter() { } { - PersistentResource child1Resource = new PersistentResource<>(child1, null, "1", badUserScope); - PersistentResource child2Resource = new PersistentResource<>(child2, null, "-2", badUserScope); - PersistentResource child3Resource = new PersistentResource<>(child3, null, "3", badUserScope); - PersistentResource child4Resource = new PersistentResource<>(child4, null, "-4", badUserScope); + RequestScope scope = new TestRequestScope(tx, badUser, dictionary); + PersistentResource child1Resource = new PersistentResource<>(child1, null, "1", scope); + PersistentResource child2Resource = new PersistentResource<>(child2, null, "-2", scope); + PersistentResource child3Resource = new PersistentResource<>(child3, null, "3", scope); + PersistentResource child4Resource = new PersistentResource<>(child4, null, "-4", scope); Set resources = Sets.newHashSet(child1Resource, child2Resource, child3Resource, child4Resource); @@ -512,7 +537,8 @@ public void testDeleteBidirectionalRelation() { left.setOne2one(right); right.setOne2one(left); - PersistentResource leftResource = new PersistentResource<>(left, null, "3", goodUserScope); + RequestScope scope = new TestRequestScope(tx, goodUser, dictionary); + PersistentResource leftResource = new PersistentResource<>(left, null, "3", scope); leftResource.deleteInverseRelation("one2one", right); @@ -525,7 +551,8 @@ public void testDeleteBidirectionalRelation() { parent.setChildren(Sets.newHashSet(child)); parent.setSpouses(Sets.newHashSet()); - PersistentResource childResource = new PersistentResource<>(child, null, "4", goodUserScope); + scope = new TestRequestScope(tx, goodUser, dictionary); + PersistentResource childResource = new PersistentResource<>(child, null, "4", scope); childResource.deleteInverseRelation("parents", parent); @@ -538,7 +565,8 @@ public void testAddBidirectionalRelation() { Left left = new Left(); Right right = new Right(); - PersistentResource leftResource = new PersistentResource<>(left, null, "3", goodUserScope); + RequestScope scope = new TestRequestScope(tx, goodUser, dictionary); + PersistentResource leftResource = new PersistentResource<>(left, null, "3", scope); leftResource.addInverseRelation("one2one", right); @@ -550,7 +578,8 @@ public void testAddBidirectionalRelation() { parent.setChildren(Sets.newHashSet()); parent.setSpouses(Sets.newHashSet()); - PersistentResource childResource = new PersistentResource<>(child, null, "4", goodUserScope); + scope = new TestRequestScope(tx, goodUser, dictionary); + PersistentResource childResource = new PersistentResource<>(child, null, "4", scope); childResource.addInverseRelation("parents", parent); @@ -560,26 +589,28 @@ public void testAddBidirectionalRelation() { @Test public void testSuccessfulOneToOneRelationshipAdd() throws Exception { - User goodUser = new User(1); - DataStoreTransaction tx = mock(DataStoreTransaction.class); Left left = new Left(); Right right = new Right(); left.setId(2); right.setId(3); +<<<<<<< HEAD RequestScope goodScope = buildRequestScope(tx, goodUser); +======= + RequestScope goodScope = new TestRequestScope(tx, goodUser, dictionary); +>>>>>>> ef111d6e... Create AggregationDataStore module (#845) PersistentResource leftResource = new PersistentResource<>(left, null, "2", goodScope); Relationship ids = new Relationship(null, new Data<>(new ResourceIdentifier("right", "3").castToResource())); - when(tx.loadObject(eq(Right.class), eq(3L), any(), any())).thenReturn(right); + when(tx.loadObject(any(), eq(3L), any())).thenReturn(right); boolean updated = leftResource.updateRelation("one2one", ids.toPersistentResources(goodScope)); goodScope.saveOrCreateObjects(); verify(tx, times(1)).save(left, goodScope); verify(tx, times(1)).save(right, goodScope); - verify(tx, times(1)).getRelation(tx, left, "one2one", Optional.empty(), Optional.empty(), Optional.empty(), - goodScope); + verify(tx, times(1)).getRelation(tx, left, getRelationship(Right.class, "one2one"), goodScope); + assertTrue(updated, "The one-2-one relationship should be added."); assertEquals(3, left.getOne2one().getId(), "The correct object was set in the one-2-one relationship"); } @@ -601,8 +632,6 @@ public void testSuccessfulOneToOneRelationshipAdd() throws Exception { */ @Test public void testSuccessfulOneToOneRelationshipAddNull() throws Exception { - User goodUser = new User(1); - DataStoreTransaction tx = mock(DataStoreTransaction.class, Answers.CALLS_REAL_METHODS); Left left = new Left(); left.setId(2); @@ -636,11 +665,12 @@ public void testSuccessfulOneToOneRelationshipAddNull() throws Exception { * final = (notMine) UNION requested */ public void testSuccessfulManyToManyRelationshipUpdate() throws Exception { - User goodUser = new User(1); - DataStoreTransaction tx = mock(DataStoreTransaction.class); - Parent parent = new Parent(); +<<<<<<< HEAD RequestScope goodScope = buildRequestScope(tx, goodUser); +======= + RequestScope goodScope = new TestRequestScope(tx, goodUser, dictionary); +>>>>>>> ef111d6e... Create AggregationDataStore module (#845) Child child1 = newChild(1); Child child2 = newChild(2); @@ -660,7 +690,7 @@ public void testSuccessfulManyToManyRelationshipUpdate() throws Exception { parent.setChildren(allChildren); parent.setSpouses(Sets.newHashSet()); - when(tx.getRelation(any(), eq(parent), eq("children"), any(), any(), any(), any())).thenReturn(allChildren); + when(tx.getRelation(any(), eq(parent), any(), any())).thenReturn(allChildren); PersistentResource parentResource = new PersistentResource<>(parent, null, "1", goodScope); @@ -670,12 +700,11 @@ public void testSuccessfulManyToManyRelationshipUpdate() throws Exception { idList.add(new ResourceIdentifier("child", "6").castToResource()); Relationship ids = new Relationship(null, new Data<>(idList)); - - when(tx.loadObject(eq(Child.class), eq(2L), any(), any())).thenReturn(child2); - when(tx.loadObject(eq(Child.class), eq(3L), any(), any())).thenReturn(child3); - when(tx.loadObject(eq(Child.class), eq(-4L), any(), any())).thenReturn(child4); - when(tx.loadObject(eq(Child.class), eq(-5L), any(), any())).thenReturn(child5); - when(tx.loadObject(eq(Child.class), eq(6L), any(), any())).thenReturn(child6); + when(tx.loadObject(any(), eq(2L), any())).thenReturn(child2); + when(tx.loadObject(any(), eq(3L), any())).thenReturn(child3); + when(tx.loadObject(any(), eq(-4L), any())).thenReturn(child4); + when(tx.loadObject(any(), eq(-5L), any())).thenReturn(child5); + when(tx.loadObject(any(), eq(6L), any())).thenReturn(child6); //Final set after operation = (3,4,5,6) Set expected = new HashSet<>(); @@ -743,7 +772,9 @@ public void testGetAttributeSuccess() { fun.setField2("blah"); fun.setField3(null); - PersistentResource funResource = new PersistentResource<>(fun, null, "1", goodUserScope); + RequestScope scope = new TestRequestScope(tx, goodUser, dictionary); + + PersistentResource funResource = new PersistentResource<>(fun, null, "1", scope); String result = (String) funResource.getAttribute("field2"); assertEquals("blah", result, "The correct attribute should be returned."); @@ -755,7 +786,9 @@ public void testGetAttributeSuccess() { public void testGetAttributeInvalidField() { FunWithPermissions fun = new FunWithPermissions(); - PersistentResource funResource = new PersistentResource<>(fun, null, "1", goodUserScope); + RequestScope scope = new TestRequestScope(tx, goodUser, dictionary); + + PersistentResource funResource = new PersistentResource<>(fun, null, "1", scope); assertThrows(InvalidAttributeException.class, () -> funResource.getAttribute("invalid")); } @@ -765,7 +798,9 @@ public void testGetAttributeInvalidFieldPermissions() { FunWithPermissions fun = new FunWithPermissions(); fun.setField1("foo"); - PersistentResource funResource = new PersistentResource<>(fun, null, "1", goodUserScope); + RequestScope scope = new TestRequestScope(tx, goodUser, dictionary); + + PersistentResource funResource = new PersistentResource<>(fun, null, "1", scope); assertThrows(ForbiddenAccessException.class, () -> funResource.getAttribute("field1")); } @@ -774,7 +809,9 @@ public void testGetAttributeInvalidFieldPermissions() { public void testGetAttributeInvalidEntityPermissions() { NoReadEntity noread = new NoReadEntity(); - PersistentResource noreadResource = new PersistentResource<>(noread, null, "1", goodUserScope); + RequestScope scope = new TestRequestScope(tx, goodUser, dictionary); + + PersistentResource noreadResource = new PersistentResource<>(noread, null, "1", scope); assertThrows(ForbiddenAccessException.class, () -> noreadResource.getAttribute("field")); } @@ -788,9 +825,10 @@ public void testGetRelationSuccess() { Set children = Sets.newHashSet(child1, child2, child3); fun.setRelation2(children); - PersistentResource funResource = new PersistentResource<>(fun, null, "3", goodUserScope); + RequestScope scope = new TestRequestScope(tx, goodUser, dictionary); + PersistentResource funResource = new PersistentResource<>(fun, null, "3", scope); - when(goodUserScope.getTransaction().getRelation(any(), eq(fun), eq("relation2"), any(), any(), any(), any())).thenReturn(children); + when(scope.getTransaction().getRelation(any(), eq(fun), any(), any())).thenReturn(children); Set results = getRelation(funResource, "relation2"); @@ -807,9 +845,10 @@ public void testGetRelationFilteredSuccess() { Set children = Sets.newHashSet(child1, child2, child3); fun.setRelation2(Sets.newHashSet(child1, child2, child3)); - PersistentResource funResource = new PersistentResource<>(fun, null, "3", goodUserScope); + RequestScope scope = new TestRequestScope(tx, goodUser, dictionary); + PersistentResource funResource = new PersistentResource<>(fun, null, "3", scope); - when(goodUserScope.getTransaction().getRelation(any(), eq(fun), eq("relation2"), any(), any(), any(), any())).thenReturn(children); + when(scope.getTransaction().getRelation(any(), eq(fun), any(), any())).thenReturn(children); Set results = getRelation(funResource, "relation2"); @@ -824,13 +863,17 @@ public void testGetRelationWithPredicateSuccess() { Child child3 = newChild(3, "chris smith"); parent.setChildren(Sets.newHashSet(child1, child2, child3)); - DataStoreTransaction tx = mock(DataStoreTransaction.class); - when(tx.getRelation(eq(tx), any(), any(), any(), any(), any(), any())).thenReturn(Sets.newHashSet(child1)); - User goodUser = new User(1); + when(tx.getRelation(eq(tx), any(), any(), any())).thenReturn(Sets.newHashSet(child1)); MultivaluedMap queryParams = new MultivaluedHashMap<>(); queryParams.add("filter[child.name]", "paul john"); +<<<<<<< HEAD RequestScope goodScope = buildRequestScope("/child", tx, goodUser, queryParams); +======= + + RequestScope goodScope = new TestRequestScope(tx, goodUser, dictionary); + +>>>>>>> ef111d6e... Create AggregationDataStore module (#845) PersistentResource parentResource = new PersistentResource<>(parent, null, "1", goodScope); @@ -850,11 +893,12 @@ public void testGetSingleRelationInMemory() { Set children = Sets.newHashSet(child1, child2, child3); parent.setChildren(children); - when(goodUserScope.getTransaction().getRelation(any(), eq(parent), eq("children"), any(), any(), any(), any())).thenReturn(children); + RequestScope scope = new TestRequestScope(tx, goodUser, dictionary); + when(scope.getTransaction().getRelation(any(), eq(parent), any(), any())).thenReturn(children); - PersistentResource parentResource = new PersistentResource<>(parent, null, "1", goodUserScope); + PersistentResource parentResource = new PersistentResource<>(parent, null, "1", scope); - PersistentResource childResource = parentResource.getRelation("children", "2"); + PersistentResource childResource = parentResource.getRelation(getRelationship(Parent.class, "children"), "2"); assertEquals("2", childResource.getId()); assertEquals("john buzzard", ((Child) childResource.getObject()).getName()); @@ -864,7 +908,9 @@ public void testGetSingleRelationInMemory() { public void testGetRelationForbiddenByEntity() { NoReadEntity noread = new NoReadEntity(); - PersistentResource noreadResource = new PersistentResource<>(noread, null, "3", goodUserScope); + RequestScope scope = new TestRequestScope(tx, badUser, dictionary); + + PersistentResource noreadResource = new PersistentResource<>(noread, null, "3", scope); assertThrows(ForbiddenAccessException.class, () -> getRelation(noreadResource, "child")); } @@ -872,7 +918,9 @@ public void testGetRelationForbiddenByEntity() { public void testGetRelationForbiddenByField() { FunWithPermissions fun = new FunWithPermissions(); - PersistentResource funResource = new PersistentResource<>(fun, null, "3", badUserScope); + RequestScope scope = new TestRequestScope(tx, badUser, dictionary); + + PersistentResource funResource = new PersistentResource<>(fun, null, "3", scope); assertThrows(ForbiddenAccessException.class, () -> getRelation(funResource, "relation1")); } @@ -881,6 +929,8 @@ public void testGetRelationForbiddenByField() { public void testGetRelationForbiddenByEntityAllowedByField() { FirstClassFields firstClassFields = new FirstClassFields(); + RequestScope badUserScope = new TestRequestScope(tx, badUser, dictionary); + PersistentResource fcResource = new PersistentResource<>(firstClassFields, null, "3", badUserScope); getRelation(fcResource, "public2"); @@ -890,6 +940,8 @@ public void testGetRelationForbiddenByEntityAllowedByField() { public void testGetAttributeForbiddenByEntityAllowedByField() { FirstClassFields firstClassFields = new FirstClassFields(); + RequestScope badUserScope = new TestRequestScope(tx, badUser, dictionary); + PersistentResource fcResource = new PersistentResource<>(firstClassFields, null, "3", badUserScope); fcResource.getAttribute("public1"); @@ -899,6 +951,8 @@ public void testGetAttributeForbiddenByEntityAllowedByField() { public void testGetRelationForbiddenByEntity2() { FirstClassFields firstClassFields = new FirstClassFields(); + RequestScope badUserScope = new TestRequestScope(tx, badUser, dictionary); + PersistentResource fcResource = new PersistentResource<>(firstClassFields, null, "3", badUserScope); assertThrows(ForbiddenAccessException.class, () -> getRelation(fcResource, "private2")); @@ -908,8 +962,10 @@ public void testGetRelationForbiddenByEntity2() { public void testGetAttributeForbiddenByEntity2() { FirstClassFields firstClassFields = new FirstClassFields(); + RequestScope scope = new TestRequestScope(tx, goodUser, dictionary); + PersistentResource fcResource = new PersistentResource<>(firstClassFields, - null, "3", goodUserScope); + null, "3", scope); assertThrows(ForbiddenAccessException.class, () -> fcResource.getAttribute("private1")); } @@ -918,7 +974,9 @@ public void testGetAttributeForbiddenByEntity2() { public void testGetRelationInvalidRelation() { FunWithPermissions fun = new FunWithPermissions(); - PersistentResource funResource = new PersistentResource<>(fun, null, "3", goodUserScope); + RequestScope scope = new TestRequestScope(tx, goodUser, dictionary); + + PersistentResource funResource = new PersistentResource<>(fun, null, "3", scope); assertThrows(InvalidAttributeException.class, () -> getRelation(funResource, "invalid")); } @@ -931,16 +989,12 @@ public void testGetRelationByIdSuccess() { Child child3 = newChild(3); fun.setRelation2(Sets.newHashSet(child1, child2, child3)); - User goodUser = new User(1); - - DataStoreTransaction tx = mock(DataStoreTransaction.class); - - when(tx.getRelation(eq(tx), any(), any(), any(), any(), any(), any())).thenReturn(Sets.newHashSet(child1)); + when(tx.getRelation(eq(tx), any(), any(), any())).thenReturn(Sets.newHashSet(child1)); RequestScope goodScope = buildRequestScope(tx, goodUser); PersistentResource funResource = new PersistentResource<>(fun, null, "3", goodScope); - PersistentResource result = funResource.getRelation("relation2", "1"); + PersistentResource result = funResource.getRelation(getRelationship(FunWithPermissions.class, "relation2"), "1"); assertEquals(1, ((Child) result.getObject()).getId(), "The correct relationship element should be returned"); } @@ -953,23 +1007,22 @@ public void testGetRelationByInvalidId() { Child child3 = newChild(3); fun.setRelation2(Sets.newHashSet(child1, child2, child3)); - User goodUser = new User(1); - - DataStoreTransaction tx = mock(DataStoreTransaction.class); - - when(tx.getRelation(eq(tx), any(), any(), any(), any(), any(), any())).thenReturn(Sets.newHashSet(child1)); + when(tx.getRelation(eq(tx), any(), any(), any())).thenReturn(Sets.newHashSet(child1)); RequestScope goodScope = buildRequestScope(tx, goodUser); PersistentResource funResource = new PersistentResource<>(fun, null, "3", goodScope); - assertThrows(InvalidObjectIdentifierException.class, () -> funResource.getRelation("relation2", "-1000")); + assertThrows(InvalidObjectIdentifierException.class, + () -> funResource.getRelation(getRelationship(FunWithPermissions.class, "relation2"), "-1000")); } @Test public void testGetRelationsNoEntityAccess() { FunWithPermissions fun = new FunWithPermissions(); - PersistentResource funResource = new PersistentResource<>(fun, null, "3", goodUserScope); + RequestScope scope = new TestRequestScope(tx, goodUser, dictionary); + + PersistentResource funResource = new PersistentResource<>(fun, null, "3", scope); Set set = getRelation(funResource, "relation4"); assertEquals(0, set.size()); @@ -979,7 +1032,9 @@ public void testGetRelationsNoEntityAccess() { public void testGetRelationsNoEntityAccess2() { FunWithPermissions fun = new FunWithPermissions(); - PersistentResource funResource = new PersistentResource<>(fun, null, "3", goodUserScope); + RequestScope scope = new TestRequestScope(tx, goodUser, dictionary); + + PersistentResource funResource = new PersistentResource<>(fun, null, "3", scope); Set set = getRelation(funResource, "relation5"); assertEquals(0, set.size()); @@ -989,9 +1044,13 @@ public void testGetRelationsNoEntityAccess2() { void testDeleteResourceSuccess() { Parent parent = newParent(1); +<<<<<<< HEAD User goodUser = new User(1); DataStoreTransaction tx = mock(DataStoreTransaction.class); RequestScope goodScope = buildRequestScope(tx, goodUser); +======= + RequestScope goodScope = new RequestScope(null, null, tx, goodUser, null, elideSettings); +>>>>>>> ef111d6e... Create AggregationDataStore module (#845) PersistentResource parentResource = new PersistentResource<>(parent, null, "1", goodScope); @@ -1009,9 +1068,13 @@ void testDeleteCascades() { invoice.setItems(Sets.newHashSet(item)); item.setInvoice(invoice); +<<<<<<< HEAD User goodUser = new User(1); DataStoreTransaction tx = mock(DataStoreTransaction.class); RequestScope goodScope = buildRequestScope(tx, goodUser); +======= + RequestScope goodScope = new RequestScope(null, null, tx, goodUser, null, elideSettings); +>>>>>>> ef111d6e... Create AggregationDataStore module (#845) PersistentResource invoiceResource = new PersistentResource<>(invoice, null, "1", goodScope); @@ -1036,9 +1099,7 @@ void testDeleteResourceUpdateRelationshipSuccess() { assertFalse(parent.getChildren().isEmpty()); - User goodUser = new User(1); - DataStoreTransaction tx = mock(DataStoreTransaction.class); - when(tx.getRelation(any(), eq(child), eq("parents"), any(), any(), any(), any())).thenReturn(parents); + when(tx.getRelation(any(), eq(child), any(), any())).thenReturn(parents); RequestScope goodScope = buildRequestScope(tx, goodUser); @@ -1058,10 +1119,14 @@ void testDeleteResourceForbidden() { NoDeleteEntity nodelete = new NoDeleteEntity(); nodelete.setId(1); +<<<<<<< HEAD DataStoreTransaction tx = mock(DataStoreTransaction.class); User goodUser = new User(1); RequestScope goodScope = buildRequestScope(tx, goodUser); +======= + RequestScope goodScope = new RequestScope(null, null, tx, goodUser, null, elideSettings); +>>>>>>> ef111d6e... Create AggregationDataStore module (#845) PersistentResource nodeleteResource = new PersistentResource<>(nodelete, null, "1", goodScope); @@ -1077,11 +1142,15 @@ void testAddRelationSuccess() { Child child = newChild(1); +<<<<<<< HEAD User goodUser = new User(1); DataStoreTransaction tx = mock(DataStoreTransaction.class); RequestScope goodScope = buildRequestScope(tx, goodUser); +======= + RequestScope goodScope = new RequestScope(null, null, tx, goodUser, null, elideSettings); +>>>>>>> ef111d6e... Create AggregationDataStore module (#845) PersistentResource funResource = new PersistentResource<>(fun, null, "3", goodScope); PersistentResource childResource = new PersistentResource<>(child, null, "1", goodScope); funResource.addRelation("relation1", childResource); @@ -1100,10 +1169,14 @@ void testAddRelationForbiddenByField() { Child child = newChild(1); +<<<<<<< HEAD User badUser = new User(-1); DataStoreTransaction tx = mock(DataStoreTransaction.class); RequestScope badScope = buildRequestScope(tx, badUser); +======= + RequestScope badScope = new RequestScope(null, null, tx, badUser, null, elideSettings); +>>>>>>> ef111d6e... Create AggregationDataStore module (#845) PersistentResource funResource = new PersistentResource<>(fun, null, "3", badScope); PersistentResource childResource = new PersistentResource<>(child, null, "1", badScope); assertThrows(ForbiddenAccessException.class, () -> funResource.addRelation("relation1", childResource)); @@ -1116,10 +1189,14 @@ void testAddRelationForbiddenByEntity() { Child child = newChild(2); noUpdate.setChildren(Sets.newHashSet()); +<<<<<<< HEAD DataStoreTransaction tx = mock(DataStoreTransaction.class); User goodUser = new User(1); RequestScope goodScope = buildRequestScope(tx, goodUser); +======= + RequestScope goodScope = new RequestScope(null, null, tx, goodUser, null, elideSettings); +>>>>>>> ef111d6e... Create AggregationDataStore module (#845) PersistentResource noUpdateResource = new PersistentResource<>(noUpdate, null, "1", goodScope); PersistentResource childResource = new PersistentResource<>(child, null, "2", goodScope); assertThrows(ForbiddenAccessException.class, () -> noUpdateResource.addRelation("children", childResource)); @@ -1131,10 +1208,14 @@ public void testAddRelationInvalidRelation() { Child child = newChild(1); +<<<<<<< HEAD User goodUser = new User(1); DataStoreTransaction tx = mock(DataStoreTransaction.class); RequestScope goodScope = buildRequestScope(tx, goodUser); +======= + RequestScope goodScope = new RequestScope(null, null, tx, goodUser, null, elideSettings); +>>>>>>> ef111d6e... Create AggregationDataStore module (#845) PersistentResource funResource = new PersistentResource<>(fun, null, "3", goodScope); PersistentResource childResource = new PersistentResource<>(child, null, "1", goodScope); assertThrows(InvalidAttributeException.class, () -> funResource.addRelation("invalid", childResource)); @@ -1148,9 +1229,13 @@ public void testRemoveToManyRelationSuccess() { Parent parent3 = newParent(3, child); child.setParents(Sets.newHashSet(parent1, parent2, parent3)); +<<<<<<< HEAD DataStoreTransaction tx = mock(DataStoreTransaction.class); User goodUser = new User(1); RequestScope goodScope = buildRequestScope(tx, goodUser); +======= + RequestScope goodScope = new RequestScope(null, null, tx, goodUser, null, elideSettings); +>>>>>>> ef111d6e... Create AggregationDataStore module (#845) PersistentResource childResource = new PersistentResource<>(child, null, "1", goodScope); PersistentResource removeResource = new PersistentResource<>(parent1, null, "1", goodScope); childResource.removeRelation("parents", removeResource); @@ -1173,9 +1258,13 @@ public void testRemoveToOneRelationSuccess() { Child child = newChild(1); fun.setRelation3(child); +<<<<<<< HEAD DataStoreTransaction tx = mock(DataStoreTransaction.class); User goodUser = new User(1); RequestScope goodScope = buildRequestScope(tx, goodUser); +======= + RequestScope goodScope = new RequestScope(null, null, tx, goodUser, null, elideSettings); +>>>>>>> ef111d6e... Create AggregationDataStore module (#845) PersistentResource funResource = new PersistentResource<>(fun, null, "1", goodScope); PersistentResource removeResource = new PersistentResource<>(child, null, "1", goodScope); @@ -1208,6 +1297,7 @@ public void testNoSaveNonModifications() { child.setReadNoAccess(secret); +<<<<<<< HEAD DataStoreTransaction tx = mock(DataStoreTransaction.class); when(tx.getRelation(any(), eq(fun), eq("relation3"), any(), any(), any(), any())).thenReturn(child); when(tx.getRelation(any(), eq(fun), eq("relation1"), any(), any(), any(), any())).thenReturn(children1); @@ -1221,6 +1311,49 @@ public void testNoSaveNonModifications() { PersistentResource childResource = new PersistentResource<>(child, null, "1", goodScope); PersistentResource secretResource = new PersistentResource<>(secret, null, "1", goodScope); PersistentResource parentResource = new PersistentResource<>(parent, null, "1", goodScope); +======= + when(tx.getRelation(any(), eq(fun), eq(com.yahoo.elide.request.Relationship.builder() + .name("relation3") + .alias("relation3") + .projection(EntityProjection.builder() + .type(Child.class) + .build()) + .build()), any())).thenReturn(child); + + when(tx.getRelation(any(), eq(fun), eq(com.yahoo.elide.request.Relationship.builder() + .name("relation1") + .alias("relation1") + .projection(EntityProjection.builder() + .type(Child.class) + .build()) + .build()), any())).thenReturn(children1); + + when(tx.getRelation(any(), eq(parent), eq(com.yahoo.elide.request.Relationship.builder() + .name("children") + .alias("children") + .projection(EntityProjection.builder() + .type(Child.class) + .build()) + .build()), any())).thenReturn(children2); + + when(tx.getRelation(any(), eq(child), eq(com.yahoo.elide.request.Relationship.builder() + .name("readNoAccess") + .alias("readNoAccess") + .projection(EntityProjection.builder() + .type(Child.class) + .build()) + .build()), any())).thenReturn(secret); + + RequestScope funScope = new TestRequestScope(tx, goodUser, dictionary); + RequestScope childScope = new TestRequestScope(tx, goodUser, dictionary); + RequestScope parentScope = new TestRequestScope(tx, goodUser, dictionary); + + + PersistentResource funResource = new PersistentResource<>(fun, null, "1", funScope); + PersistentResource childResource = new PersistentResource<>(child, null, "1", childScope); + PersistentResource secretResource = new PersistentResource<>(secret, null, "1", childScope); + PersistentResource parentResource = new PersistentResource<>(parent, null, "1", parentScope); +>>>>>>> ef111d6e... Create AggregationDataStore module (#845) // Add an existing to-one relationship funResource.addRelation("relation3", childResource); @@ -1252,11 +1385,13 @@ public void testNoSaveNonModifications() { // Clear empty to-one relation secretResource.clearRelation("readNoAccess"); - goodScope.saveOrCreateObjects(); - verify(tx, never()).save(fun, goodScope); - verify(tx, never()).save(child, goodScope); - verify(tx, never()).save(parent, goodScope); - verify(tx, never()).save(secret, goodScope); + parentScope.saveOrCreateObjects(); + childScope.saveOrCreateObjects(); + funScope.saveOrCreateObjects(); + verify(tx, never()).save(fun, funScope); + verify(tx, never()).save(child, childScope); + verify(tx, never()).save(parent, parentScope); + verify(tx, never()).save(secret, childScope); } @Test() @@ -1266,9 +1401,13 @@ public void testRemoveNonexistingToOneRelation() { Child unownedChild = newChild(2); fun.setRelation3(ownedChild); +<<<<<<< HEAD DataStoreTransaction tx = mock(DataStoreTransaction.class); User goodUser = new User(1); RequestScope goodScope = buildRequestScope(tx, goodUser); +======= + RequestScope goodScope = new RequestScope(null, null, tx, goodUser, null, elideSettings); +>>>>>>> ef111d6e... Create AggregationDataStore module (#845) PersistentResource funResource = new PersistentResource<>(fun, null, "1", goodScope); PersistentResource removeResource = new PersistentResource<>(unownedChild, null, "1", goodScope); @@ -1291,9 +1430,13 @@ public void testRemoveNonexistingToManyRelation() { Parent unownedParent = newParent(4, null); +<<<<<<< HEAD DataStoreTransaction tx = mock(DataStoreTransaction.class); User goodUser = new User(1); RequestScope goodScope = buildRequestScope(tx, goodUser); +======= + RequestScope goodScope = new RequestScope(null, null, tx, goodUser, null, elideSettings); +>>>>>>> ef111d6e... Create AggregationDataStore module (#845) PersistentResource childResource = new PersistentResource<>(child, null, "1", goodScope); PersistentResource removeResource = new PersistentResource<>(unownedParent, null, "1", goodScope); childResource.removeRelation("parents", removeResource); @@ -1318,11 +1461,21 @@ public void testClearToManyRelationSuccess() { Set parents = Sets.newHashSet(parent1, parent2, parent3); child.setParents(parents); - DataStoreTransaction tx = mock(DataStoreTransaction.class); - when(tx.getRelation(any(), eq(child), eq("parents"), any(), any(), any(), any())).thenReturn(parents); + when(tx.getRelation(any(), eq(child), any(), any())).thenReturn(parents); +<<<<<<< HEAD User goodUser = new User(1); RequestScope goodScope = buildRequestScope(tx, goodUser); +======= + RequestScope goodScope = new RequestScope(null, null, tx, goodUser, null, elideSettings); + goodScope.setEntityProjection(EntityProjection.builder() + .type(Child.class) + .relationship("parents", + EntityProjection.builder() + .type(Parent.class) + .build()) + .build()); +>>>>>>> ef111d6e... Create AggregationDataStore module (#845) PersistentResource childResource = new PersistentResource<>(child, null, "1", goodScope); @@ -1346,12 +1499,24 @@ public void testClearToOneRelationSuccess() { Child child = newChild(1); fun.setRelation3(child); - DataStoreTransaction tx = mock(DataStoreTransaction.class); + when(tx.getRelation(any(), eq(fun), any(), any())).thenReturn(child); +<<<<<<< HEAD when(tx.getRelation(any(), eq(fun), eq("relation3"), any(), any(), any(), any())).thenReturn(child); User goodUser = new User(1); RequestScope goodScope = buildRequestScope(tx, goodUser); +======= + RequestScope goodScope = new RequestScope(null, null, tx, goodUser, null, elideSettings); + goodScope.setEntityProjection(EntityProjection.builder() + .type(FunWithPermissions.class) + .relationship("relation3", + EntityProjection.builder() + .type(Child.class) + .build()) + .build()); + +>>>>>>> ef111d6e... Create AggregationDataStore module (#845) PersistentResource funResource = new PersistentResource<>(fun, null, "1", goodScope); funResource.clearRelation("relation3"); @@ -1364,11 +1529,17 @@ public void testClearToOneRelationSuccess() { @Test() public void testClearRelationFilteredByReadAccess() { - User goodUser = new User(1); - DataStoreTransaction tx = mock(DataStoreTransaction.class); Parent parent = new Parent(); RequestScope goodScope = buildRequestScope(tx, goodUser); + goodScope.setEntityProjection(EntityProjection.builder() + .type(Parent.class) + .relationship("children", + EntityProjection.builder() + .type(Child.class) + .build()) + .build()); + Child child1 = newChild(1); Child child2 = newChild(2); Child child3 = newChild(3); @@ -1377,7 +1548,6 @@ public void testClearRelationFilteredByReadAccess() { Child child5 = newChild(-5); child5.setId(-5); //Not accessible to goodUser - //All = (1,2,3,4,5) //Mine = (1,2,3) Set allChildren = new HashSet<>(); @@ -1389,7 +1559,7 @@ public void testClearRelationFilteredByReadAccess() { parent.setChildren(allChildren); parent.setSpouses(Sets.newHashSet()); - when(tx.getRelation(any(), eq(parent), eq("children"), any(), any(), any(), any())).thenReturn(allChildren); + when(tx.getRelation(any(), eq(parent), any(), any())).thenReturn(allChildren); PersistentResource parentResource = new PersistentResource<>(parent, null, "1", goodScope); @@ -1427,9 +1597,22 @@ public void testClearRelationInvalidToOneUpdatePermission() { left.setNoUpdateOne2One(right); right.setNoUpdateOne2One(left); +<<<<<<< HEAD DataStoreTransaction tx = mock(DataStoreTransaction.class); User goodUser = new User(1); RequestScope goodScope = buildRequestScope(tx, goodUser); +======= + RequestScope goodScope = new RequestScope(null, null, tx, goodUser, null, elideSettings); + + goodScope.setEntityProjection(EntityProjection.builder() + .type(Left.class) + .relationship("noUpdateOne2One", + EntityProjection.builder() + .type(Right.class) + .build()) + .build()); + +>>>>>>> ef111d6e... Create AggregationDataStore module (#845) PersistentResource leftResource = new PersistentResource<>(left, null, "1", goodScope); assertThrows( @@ -1448,9 +1631,13 @@ public void testNoChangeRelationInvalidToOneUpdatePermission() { left.setNoUpdateOne2One(right); right.setNoUpdateOne2One(left); +<<<<<<< HEAD DataStoreTransaction tx = mock(DataStoreTransaction.class); User goodUser = new User(1); RequestScope goodScope = buildRequestScope(tx, goodUser); +======= + RequestScope goodScope = new RequestScope(null, null, tx, goodUser, null, elideSettings); +>>>>>>> ef111d6e... Create AggregationDataStore module (#845) PersistentResource leftResource = new PersistentResource<>(left, null, "1", goodScope); assertThrows( @@ -1474,12 +1661,24 @@ public void testClearRelationInvalidToManyUpdatePermission() { right1.setNoUpdate(Sets.newHashSet(left)); right2.setNoUpdate(Sets.newHashSet(left)); - DataStoreTransaction tx = mock(DataStoreTransaction.class); + when(tx.getRelation(any(), eq(left), any(), any())).thenReturn(noInverseUpdate); +<<<<<<< HEAD when(tx.getRelation(any(), eq(left), eq("noInverseUpdate"), any(), any(), any(), any())).thenReturn(noInverseUpdate); User goodUser = new User(1); RequestScope goodScope = buildRequestScope(tx, goodUser); +======= + RequestScope goodScope = new RequestScope(null, null, tx, goodUser, null, elideSettings); + goodScope.setEntityProjection(EntityProjection.builder() + .type(Left.class) + .relationship("noInverseUpdate", + EntityProjection.builder() + .type(Right.class) + .build()) + .build()); + +>>>>>>> ef111d6e... Create AggregationDataStore module (#845) PersistentResource leftResource = new PersistentResource<>(left, null, "1", goodScope); assertThrows( @@ -1497,11 +1696,24 @@ public void testClearRelationInvalidToOneDeletePermission() { noDelete.setId(1); left.setNoDeleteOne2One(noDelete); +<<<<<<< HEAD DataStoreTransaction tx = mock(DataStoreTransaction.class); when(tx.getRelation(any(), eq(left), eq("noDeleteOne2One"), any(), any(), any(), any())).thenReturn(noDelete); User goodUser = new User(1); RequestScope goodScope = buildRequestScope(tx, goodUser); +======= + when(tx.getRelation(any(), eq(left), any(), any())).thenReturn(noDelete); + RequestScope goodScope = new RequestScope(null, null, tx, goodUser, null, elideSettings); + goodScope.setEntityProjection(EntityProjection.builder() + .type(Left.class) + .relationship("noDeleteOne2One", + EntityProjection.builder() + .type(NoDeleteEntity.class) + .build()) + .build()); + +>>>>>>> ef111d6e... Create AggregationDataStore module (#845) PersistentResource leftResource = new PersistentResource<>(left, null, "1", goodScope); assertTrue(leftResource.clearRelation("noDeleteOne2One")); assertNull(leftResource.getObject().getNoDeleteOne2One()); @@ -1514,9 +1726,13 @@ public void testClearRelationInvalidRelation() { Child child = newChild(1); fun.setRelation3(child); +<<<<<<< HEAD DataStoreTransaction tx = mock(DataStoreTransaction.class); User goodUser = new User(1); RequestScope goodScope = buildRequestScope(tx, goodUser); +======= + RequestScope goodScope = new RequestScope(null, null, tx, goodUser, null, elideSettings); +>>>>>>> ef111d6e... Create AggregationDataStore module (#845) PersistentResource funResource = new PersistentResource<>(fun, null, "1", goodScope); assertThrows(InvalidAttributeException.class, () -> funResource.clearRelation("invalid")); } @@ -1525,10 +1741,14 @@ public void testClearRelationInvalidRelation() { public void testUpdateAttributeSuccess() { Parent parent = newParent(1); +<<<<<<< HEAD DataStoreTransaction tx = mock(DataStoreTransaction.class); User goodUser = new User(1); RequestScope goodScope = buildRequestScope(tx, goodUser); +======= + RequestScope goodScope = new RequestScope(null, null, tx, goodUser, null, elideSettings); +>>>>>>> ef111d6e... Create AggregationDataStore module (#845) PersistentResource parentResource = new PersistentResource<>(parent, null, "1", goodScope); parentResource.updateAttribute("firstName", "foobar"); @@ -1542,10 +1762,14 @@ public void testUpdateAttributeSuccess() { public void testUpdateAttributeInvalidAttribute() { Parent parent = newParent(1); +<<<<<<< HEAD DataStoreTransaction tx = mock(DataStoreTransaction.class); User goodUser = new User(1); RequestScope goodScope = buildRequestScope(tx, goodUser); +======= + RequestScope goodScope = new RequestScope(null, null, tx, goodUser, null, elideSettings); +>>>>>>> ef111d6e... Create AggregationDataStore module (#845) PersistentResource parentResource = new PersistentResource<>(parent, null, "1", goodScope); assertThrows(InvalidAttributeException.class, () -> parentResource.updateAttribute("invalid", "foobar")); } @@ -1555,11 +1779,15 @@ public void testUpdateAttributeInvalidUpdatePermission() { FunWithPermissions fun = new FunWithPermissions(); fun.setId(1); +<<<<<<< HEAD DataStoreTransaction tx = mock(DataStoreTransaction.class); User badUser = new User(-1); RequestScope badScope = buildRequestScope(tx, badUser); +======= + RequestScope badScope = new RequestScope(null, null, tx, badUser, null, elideSettings); +>>>>>>> ef111d6e... Create AggregationDataStore module (#845) PersistentResource funResource = new PersistentResource<>(fun, null, "1", badScope); @@ -1575,10 +1803,14 @@ public void testUpdateAttributeInvalidUpdatePermissionNoChange() { FunWithPermissions fun = new FunWithPermissions(); fun.setId(1); +<<<<<<< HEAD DataStoreTransaction tx = mock(DataStoreTransaction.class); User badUser = new User(-1); RequestScope badScope = buildRequestScope(tx, badUser); +======= + RequestScope badScope = new RequestScope(null, null, tx, badUser, null, elideSettings); +>>>>>>> ef111d6e... Create AggregationDataStore module (#845) PersistentResource funResource = new PersistentResource<>(fun, null, "1", badScope); @@ -1597,15 +1829,26 @@ public void testLoadRecords() { Child child4 = newChild(4); Child child5 = newChild(5); - DataStoreTransaction tx = mock(DataStoreTransaction.class); - User goodUser = new User(1); + EntityProjection collection = EntityProjection.builder() + .type(Child.class) - when(tx.loadObjects(eq(Child.class), any(), any(), any(), any(RequestScope.class))) + .build(); + + when(tx.loadObjects(eq(collection), any(RequestScope.class))) .thenReturn(Lists.newArrayList(child1, child2, child3, child4, child5)); +<<<<<<< HEAD RequestScope goodScope = buildRequestScope(tx, goodUser); Set loaded = PersistentResource.loadRecords(Child.class, new ArrayList<>(), Optional.empty(), Optional.empty(), Optional.empty(), goodScope); +======= + RequestScope goodScope = new RequestScope(null, null, tx, goodUser, null, elideSettings); + goodScope.setEntityProjection(collection); + + Set loaded = PersistentResource.loadRecords(EntityProjection.builder() + .type(Child.class) + .build(), new ArrayList<>(), goodScope); +>>>>>>> ef111d6e... Create AggregationDataStore module (#845) Set expected = Sets.newHashSet(child1, child4, child5); @@ -1623,55 +1866,84 @@ public void testLoadRecords() { public void testLoadRecordSuccess() { Child child1 = newChild(1); - DataStoreTransaction tx = mock(DataStoreTransaction.class); - User goodUser = new User(1); + EntityProjection collection = EntityProjection.builder() + .type(Child.class) - when(tx.loadObject(eq(Child.class), eq(1L), any(), any())).thenReturn(child1); + .build(); + when(tx.loadObject(eq(collection), eq(1L), any())).thenReturn(child1); + +<<<<<<< HEAD RequestScope goodScope = buildRequestScope(tx, goodUser); PersistentResource loaded = PersistentResource.loadRecord(Child.class, "1", goodScope); +======= + RequestScope goodScope = new RequestScope(null, null, tx, goodUser, null, elideSettings); + goodScope.setEntityProjection(collection); + PersistentResource loaded = PersistentResource.loadRecord(EntityProjection.builder() + .type(Child.class) + .build(), "1", goodScope); +>>>>>>> ef111d6e... Create AggregationDataStore module (#845) assertEquals(child1, loaded.getObject(), "The load function should return the requested child object"); } @Test public void testLoadRecordInvalidId() { - DataStoreTransaction tx = mock(DataStoreTransaction.class); - User goodUser = new User(1); + EntityProjection collection = EntityProjection.builder() + .type(Child.class) + + .build(); - when(tx.loadObject(eq(Child.class), eq("1"), any(), any())).thenReturn(null); + when(tx.loadObject(eq(collection), eq("1"), any())).thenReturn(null); +<<<<<<< HEAD RequestScope goodScope = buildRequestScope(tx, goodUser); +======= + RequestScope goodScope = new RequestScope(null, null, tx, goodUser, null, elideSettings); + goodScope.setEntityProjection(collection); +>>>>>>> ef111d6e... Create AggregationDataStore module (#845) assertThrows( InvalidObjectIdentifierException.class, - () -> PersistentResource.loadRecord(Child.class, "1", goodScope)); + () -> PersistentResource.loadRecord(EntityProjection.builder() + + .type(Child.class) + .build(), "1", goodScope)); } @Test public void testLoadRecordForbidden() { NoReadEntity noRead = new NoReadEntity(); noRead.setId(1); - DataStoreTransaction tx = mock(DataStoreTransaction.class); - User goodUser = new User(1); + EntityProjection collection = EntityProjection.builder() + .type(NoReadEntity.class) - when(tx.loadObject(eq(NoReadEntity.class), eq(1L), any(), any())).thenReturn(noRead); + .build(); + when(tx.loadObject(eq(collection), eq(1L), any())).thenReturn(noRead); + +<<<<<<< HEAD RequestScope goodScope = buildRequestScope(tx, goodUser); assertThrows( ForbiddenAccessException.class, () -> PersistentResource.loadRecord(NoReadEntity.class, "1", goodScope)); +======= + RequestScope goodScope = new RequestScope(null, null, tx, goodUser, null, elideSettings); + goodScope.setEntityProjection(collection); +>>>>>>> ef111d6e... Create AggregationDataStore module (#845) } @Test() public void testCreateObjectSuccess() { Parent parent = newParent(1); - DataStoreTransaction tx = mock(DataStoreTransaction.class); - User goodUser = new User(1); - when(tx.createNewObject(Parent.class)).thenReturn(parent); +<<<<<<< HEAD RequestScope goodScope = buildRequestScope(tx, goodUser); PersistentResource created = PersistentResource.createObject(null, Parent.class, goodScope, Optional.of("uuid")); +======= + RequestScope goodScope = new RequestScope(null, null, tx, goodUser, null, elideSettings); + PersistentResource created = PersistentResource.createObject(Parent.class, goodScope, Optional.of("uuid")); +>>>>>>> ef111d6e... Create AggregationDataStore module (#845) parent.setChildren(new HashSet<>()); created.getRequestScope().getPermissionExecutor().executeCommitChecks(); @@ -1687,11 +1959,15 @@ public void testCreateMappedIdObjectSuccess() { job.setTitle("day job"); job.setParent(newParent(1)); - final DataStoreTransaction tx = mock(DataStoreTransaction.class); when(tx.createNewObject(Job.class)).thenReturn(job); +<<<<<<< HEAD final RequestScope goodScope = buildRequestScope(tx, new User(1)); PersistentResource created = PersistentResource.createObject(null, Job.class, goodScope, Optional.empty()); +======= + final RequestScope goodScope = new RequestScope(null, null, tx, new User(1), null, elideSettings); + PersistentResource created = PersistentResource.createObject(Job.class, goodScope, Optional.empty()); +>>>>>>> ef111d6e... Create AggregationDataStore module (#845) created.getRequestScope().getPermissionExecutor().executeCommitChecks(); assertEquals("day job", created.getObject().getTitle(), @@ -1699,7 +1975,7 @@ public void testCreateMappedIdObjectSuccess() { ); assertNull(created.getObject().getJobId(), "The create function should not override the ID"); - created = PersistentResource.createObject(null, Job.class, goodScope, Optional.of("1234")); + created = PersistentResource.createObject(Job.class, goodScope, Optional.of("1234")); created.getRequestScope().getPermissionExecutor().executeCommitChecks(); assertEquals("day job", created.getObject().getTitle(), @@ -1712,8 +1988,6 @@ public void testCreateMappedIdObjectSuccess() { public void testCreateObjectForbidden() { NoCreateEntity noCreate = new NoCreateEntity(); noCreate.setId(1); - DataStoreTransaction tx = mock(DataStoreTransaction.class); - User goodUser = new User(1); when(tx.createNewObject(NoCreateEntity.class)).thenReturn(noCreate); @@ -1722,7 +1996,7 @@ public void testCreateObjectForbidden() { assertThrows( ForbiddenAccessException.class, () -> { - PersistentResource created = PersistentResource.createObject(null, NoCreateEntity.class, goodScope, Optional.of("1")); + PersistentResource created = PersistentResource.createObject(NoCreateEntity.class, goodScope, Optional.of("1")); created.getRequestScope().getPermissionExecutor().executeCommitChecks(); } ); @@ -1740,9 +2014,7 @@ public void testDeletePermissionCheckedOnInverseRelationship() { right.setAllowDeleteAtFieldLevel(Sets.newHashSet(left)); //Bad User triggers the delete permission failure - User badUser = new User(-1); - DataStoreTransaction tx = mock(DataStoreTransaction.class); - when(tx.getRelation(any(), eq(left), eq("fieldLevelDelete"), any(), any(), any(), any())).thenReturn(rights); + when(tx.getRelation(any(), eq(left), any(), any())).thenReturn(rights); RequestScope badScope = buildRequestScope(tx, badUser); PersistentResource leftResource = new PersistentResource<>(left, null, badScope.getUUIDFor(left), badScope); @@ -1765,9 +2037,7 @@ public void testUpdatePermissionCheckedOnInverseRelationship() { List empty = new ArrayList<>(); Relationship ids = new Relationship(null, new Data<>(empty)); - User goodUser = new User(1); - DataStoreTransaction tx = mock(DataStoreTransaction.class); - when(tx.getRelation(any(), eq(left), eq("noInverseUpdate"), any(), any(), any(), any())).thenReturn(rights); + when(tx.getRelation(any(), eq(left), any(), any())).thenReturn(rights); RequestScope goodScope = buildRequestScope(tx, goodUser); PersistentResource leftResource = new PersistentResource<>(left, null, goodScope.getUUIDFor(left), goodScope); @@ -1785,7 +2055,6 @@ public void testFieldLevelAudit() throws Exception { Parent parent = newParent(7); - User goodUser = new User(1); TestAuditLogger logger = new TestAuditLogger(); RequestScope requestScope = getUserScope(goodUser, logger); PersistentResource parentResource = new PersistentResource<>(parent, null, requestScope.getUUIDFor(parent), requestScope); @@ -1806,7 +2075,6 @@ public void testClassLevelAudit() throws Exception { Child child = newChild(5); Parent parent = newParent(7); - User goodUser = new User(1); TestAuditLogger logger = new TestAuditLogger(); RequestScope requestScope = getUserScope(goodUser, logger); PersistentResource parentResource = new PersistentResource<>( @@ -1829,9 +2097,7 @@ public void testOwningRelationshipInverseUpdates() { Parent parent = newParent(1); Child child = newChild(2); - User goodUser = new User(1); - DataStoreTransaction tx = mock(DataStoreTransaction.class); - when(tx.getRelation(any(), eq(parent), eq("children"), any(), any(), any(), any())).thenReturn(parent.getChildren()); + when(tx.getRelation(any(), eq(parent), any(), any())).thenReturn(parent.getChildren()); RequestScope goodScope = buildRequestScope(tx, goodUser); @@ -1852,7 +2118,7 @@ public void testOwningRelationshipInverseUpdates() { assertTrue(child.getParents().contains(parent), "The non-owning relationship should also be updated"); reset(tx); - when(tx.getRelation(any(), eq(parent), eq("children"), any(), any(), any(), any())).thenReturn(parent.getChildren()); + when(tx.getRelation(any(), eq(parent), any(), any())).thenReturn(parent.getChildren()); parentResource.clearRelation("children"); @@ -1866,13 +2132,16 @@ public void testOwningRelationshipInverseUpdates() { @Test public void testIsIdGenerated() { + RequestScope scope = new TestRequestScope(tx, goodUser, dictionary); - PersistentResource generated = new PersistentResource<>(new Child(), null, "1", goodUserScope); + PersistentResource generated = new PersistentResource<>(new Child(), null, "1", scope); assertTrue(generated.isIdGenerated(), "isIdGenerated returns true when ID field has the GeneratedValue annotation"); - PersistentResource notGenerated = new PersistentResource<>(new NoCreateEntity(), null, "1", goodUserScope); + scope = new TestRequestScope(tx, goodUser, dictionary); + + PersistentResource notGenerated = new PersistentResource<>(new NoCreateEntity(), null, "1", scope); assertFalse(notGenerated.isIdGenerated(), "isIdGenerated returns false when ID field does not have the GeneratedValue annotation"); @@ -1890,11 +2159,20 @@ public void testSharePermissionErrorOnUpdateSingularRelationship() { idList.add(new ResourceIdentifier("noshare", "1").castToResource()); Relationship ids = new Relationship(null, new Data<>(idList)); - User goodUser = new User(1); - DataStoreTransaction tx = mock(DataStoreTransaction.class); - when(tx.loadObject(eq(NoShareEntity.class), eq(1L), any(), any())).thenReturn(noShare); + EntityProjection collection = EntityProjection.builder() + .type(NoShareEntity.class) + + .build(); + + when(tx.loadObject(eq(collection), eq(1L), any())).thenReturn(noShare); + RequestScope goodScope = new TestRequestScope(tx, goodUser, dictionary); + + +<<<<<<< HEAD RequestScope goodScope = buildRequestScope(tx, goodUser); +======= +>>>>>>> ef111d6e... Create AggregationDataStore module (#845) PersistentResource userResource = new PersistentResource<>(userModel, null, goodScope.getUUIDFor(userModel), goodScope); assertThrows( @@ -1913,11 +2191,14 @@ public void testSharePermissionErrorOnUpdateRelationshipPackageLevel() { unShareableList.add(new ResourceIdentifier("unshareableWithEntityUnshare", "1").castToResource()); Relationship unShareales = new Relationship(null, new Data<>(unShareableList)); - User goodUser = new User(1); - DataStoreTransaction tx = mock(DataStoreTransaction.class); - when(tx.loadObject(eq(UnshareableWithEntityUnshare.class), eq(1L), any(), any())).thenReturn(unshareableWithEntityUnshare); + when(tx.loadObject(any(), eq(1L), any())).thenReturn(unshareableWithEntityUnshare); + + RequestScope goodScope = new TestRequestScope(tx, goodUser, dictionary); +<<<<<<< HEAD RequestScope goodScope = buildRequestScope(tx, goodUser); +======= +>>>>>>> ef111d6e... Create AggregationDataStore module (#845) PersistentResource containerResource = new PersistentResource<>(containerWithPackageShare, null, goodScope.getUUIDFor(containerWithPackageShare), goodScope); assertThrows( @@ -1937,11 +2218,14 @@ public void testSharePermissionSuccessOnUpdateManyRelationshipPackageLevel() { shareableList.add(new ResourceIdentifier("shareableWithPackageShare", "1").castToResource()); Relationship shareables = new Relationship(null, new Data<>(shareableList)); - User goodUser = new User(1); - DataStoreTransaction tx = mock(DataStoreTransaction.class); - when(tx.loadObject(eq(ShareableWithPackageShare.class), eq(1L), any(), any())).thenReturn(shareableWithPackageShare); + when(tx.loadObject(any(), eq(1L), any())).thenReturn(shareableWithPackageShare); + + RequestScope goodScope = new TestRequestScope(tx, goodUser, dictionary); +<<<<<<< HEAD RequestScope goodScope = buildRequestScope(tx, goodUser); +======= +>>>>>>> ef111d6e... Create AggregationDataStore module (#845) PersistentResource containerResource = new PersistentResource<>(containerWithPackageShare, null, goodScope.getUUIDFor(containerWithPackageShare), goodScope); containerResource.updateRelation("shareableWithPackageShares", shareables.toPersistentResources(goodScope)); @@ -1965,12 +2249,15 @@ public void testSharePermissionErrorOnUpdateManyRelationship() { idList.add(new ResourceIdentifier("noshare", "2").castToResource()); Relationship ids = new Relationship(null, new Data<>(idList)); - User goodUser = new User(1); - DataStoreTransaction tx = mock(DataStoreTransaction.class); - when(tx.loadObject(eq(NoShareEntity.class), eq(1L), any(), any())).thenReturn(noShare1); - when(tx.loadObject(eq(NoShareEntity.class), eq(2L), any(), any())).thenReturn(noShare2); + when(tx.loadObject(any(), eq(1L), any())).thenReturn(noShare1); + when(tx.loadObject(any(), eq(2L), any())).thenReturn(noShare2); + + RequestScope goodScope = new TestRequestScope(tx, goodUser, dictionary); +<<<<<<< HEAD RequestScope goodScope = buildRequestScope(tx, goodUser); +======= +>>>>>>> ef111d6e... Create AggregationDataStore module (#845) PersistentResource userResource = new PersistentResource<>(userModel, null, goodScope.getUUIDFor(userModel), goodScope); assertThrows( @@ -1996,12 +2283,15 @@ public void testSharePermissionSuccessOnUpdateManyRelationship() { idList.add(new ResourceIdentifier("noshare", "1").castToResource()); Relationship ids = new Relationship(null, new Data<>(idList)); - User goodUser = new User(1); - DataStoreTransaction tx = mock(DataStoreTransaction.class); - when(tx.loadObject(eq(NoShareEntity.class), eq(1L), any(), any())).thenReturn(noShare1); - when(tx.getRelation(any(), eq(userModel), eq("noShares"), any(), any(), any(), any())).thenReturn(noshares); + RequestScope goodScope = new TestRequestScope(tx, goodUser, dictionary); + + when(tx.loadObject(any(), eq(1L), any())).thenReturn(noShare1); + when(tx.getRelation(any(), eq(userModel), any(), any())).thenReturn(noshares); +<<<<<<< HEAD RequestScope goodScope = buildRequestScope(tx, goodUser); +======= +>>>>>>> ef111d6e... Create AggregationDataStore module (#845) PersistentResource userResource = new PersistentResource<>(userModel, null, goodScope.getUUIDFor(userModel), goodScope); boolean returnVal = userResource.updateRelation("noShares", ids.toPersistentResources(goodScope)); @@ -2025,13 +2315,16 @@ public void testSharePermissionSuccessOnUpdateSingularRelationship() { idList.add(new ResourceIdentifier("noshare", "1").castToResource()); Relationship ids = new Relationship(null, new Data<>(idList)); - User goodUser = new User(1); - DataStoreTransaction tx = mock(DataStoreTransaction.class); + when(tx.getRelation(any(), eq(userModel), any(), any())).thenReturn(noShare); + when(tx.loadObject(any(), eq(1L), any())).thenReturn(noShare); + + RequestScope goodScope = new TestRequestScope(tx, goodUser, dictionary); - when(tx.getRelation(any(), eq(userModel), eq("noShare"), any(), any(), any(), any())).thenReturn(noShare); - when(tx.loadObject(eq(NoShareEntity.class), eq(1L), any(), any())).thenReturn(noShare); +<<<<<<< HEAD RequestScope goodScope = buildRequestScope(tx, goodUser); +======= +>>>>>>> ef111d6e... Create AggregationDataStore module (#845) PersistentResource userResource = new PersistentResource<>(userModel, null, goodScope.getUUIDFor(userModel), goodScope); boolean returnVal = userResource.updateRelation("noShare", ids.toPersistentResources(goodScope)); @@ -2053,11 +2346,14 @@ public void testSharePermissionSuccessOnClearSingularRelationship() { List empty = new ArrayList<>(); Relationship ids = new Relationship(null, new Data<>(empty)); - User goodUser = new User(1); - DataStoreTransaction tx = mock(DataStoreTransaction.class); - when(tx.getRelation(any(), eq(userModel), eq("noShare"), any(), any(), any(), any())).thenReturn(noShare); + when(tx.getRelation(any(), eq(userModel), any(), any())).thenReturn(noShare); + + RequestScope goodScope = new TestRequestScope(tx, goodUser, dictionary); +<<<<<<< HEAD RequestScope goodScope = buildRequestScope(tx, goodUser); +======= +>>>>>>> ef111d6e... Create AggregationDataStore module (#845) PersistentResource userResource = new PersistentResource<>(userModel, null, goodScope.getUUIDFor(userModel), goodScope); boolean returnVal = userResource.updateRelation("noShare", ids.toPersistentResources(goodScope)); @@ -2077,7 +2373,6 @@ public void testCollectionChangeSpecType() { return condFn.apply((Collection) spec.getOriginal(), (Collection) spec.getModified()); }; - DataStoreTransaction tx = mock(DataStoreTransaction.class); // Ensure that change specs coming from collections work properly ChangeSpecModel csModel = new ChangeSpecModel((spec) -> collectionCheck @@ -2086,7 +2381,7 @@ public void testCollectionChangeSpecType() { PersistentResource model = bootstrapPersistentResource(csModel, tx); - when(tx.getRelation(any(), eq(model.obj), eq("otherKids"), any(), any(), any(), any())).thenReturn(new HashSet<>()); + when(tx.getRelation(any(), eq(model.obj), any(), any())).thenReturn(new HashSet<>()); /* Attributes */ // Set new data from null @@ -2124,7 +2419,8 @@ public void testCollectionChangeSpecType() { model.getObject().checkFunction = (spec) -> collectionCheck.apply("otherKids").apply(spec, (original, modified) -> original.size() == 3 && original.contains(new ChangeSpecChild(1)) && original.contains(new ChangeSpecChild(2)) && original.contains(new ChangeSpecChild(3)) && modified.size() == 2 && modified.contains(new ChangeSpecChild(1)) && modified.contains(new ChangeSpecChild(3))); model.removeRelation("otherKids", bootstrapPersistentResource(child2)); - when(tx.getRelation(any(), eq(model.obj), eq("otherKids"), any(), any(), any(), any())).thenReturn(Sets.newHashSet(child1, child3)); + when(tx.getRelation(any(), eq(model.obj), any(), any())).thenReturn(Sets.newHashSet(child1, child3)); + // Clear the rest model.getObject().checkFunction = (spec) -> collectionCheck.apply("otherKids").apply(spec, (original, modified) -> original.size() <= 2 && modified.size() < original.size()); @@ -2168,24 +2464,23 @@ public void testRelationChangeSpecType() { } return checkFn.apply((ChangeSpecChild) spec.getOriginal(), (ChangeSpecChild) spec.getModified()); }; - DataStoreTransaction tx = mock(DataStoreTransaction.class); PersistentResource model = bootstrapPersistentResource(new ChangeSpecModel((spec) -> relCheck.apply(spec, (original, modified) -> (original == null) && new ChangeSpecChild(1).equals(modified))), tx); - when(tx.getRelation(any(), eq(model.obj), eq("child"), any(), any(), any(), any())).thenReturn(null); + when(tx.getRelation(any(), eq(model.obj), any(), any())).thenReturn(null); ChangeSpecChild child1 = new ChangeSpecChild(1); assertTrue(model.updateRelation("child", Sets.newHashSet(bootstrapPersistentResource(child1, tx)))); - when(tx.getRelation(any(), eq(model.obj), eq("child"), any(), any(), any(), any())).thenReturn(child1); + when(tx.getRelation(any(), eq(model.obj), any(), any())).thenReturn(child1); model.getObject().checkFunction = (spec) -> relCheck.apply(spec, (original, modified) -> new ChangeSpecChild(1).equals(original) && new ChangeSpecChild(2).equals(modified)); ChangeSpecChild child2 = new ChangeSpecChild(2); assertTrue(model.updateRelation("child", Sets.newHashSet(bootstrapPersistentResource(child2, tx)))); - when(tx.getRelation(any(), eq(model.obj), eq("child"), any(), any(), any(), any())).thenReturn(child2); + when(tx.getRelation(any(), eq(model.obj), any(), any())).thenReturn(child2); model.getObject().checkFunction = (spec) -> relCheck.apply(spec, (original, modified) -> new ChangeSpecChild(2).equals(original) && modified == null); assertTrue(model.updateRelation("child", null)); @@ -2250,10 +2545,12 @@ public void testEqualsAndHashcode() { Child childWithId = newChild(1); Child childWithoutId = newChild(0); - PersistentResource resourceWithId = new PersistentResource<>(childWithId, null, goodUserScope.getUUIDFor(childWithId), goodUserScope); - PersistentResource resourceWithDifferentId = new PersistentResource<>(childWithoutId, null, goodUserScope.getUUIDFor(childWithoutId), goodUserScope); - PersistentResource resourceWithUUID = new PersistentResource<>(childWithoutId, null, "abc", goodUserScope); - PersistentResource resourceWithIdAndUUID = new PersistentResource<>(childWithId, null, "abc", goodUserScope); + RequestScope scope = new TestRequestScope(tx, goodUser, dictionary); + + PersistentResource resourceWithId = new PersistentResource<>(childWithId, null, scope.getUUIDFor(childWithId), scope); + PersistentResource resourceWithDifferentId = new PersistentResource<>(childWithoutId, null, scope.getUUIDFor(childWithoutId), scope); + PersistentResource resourceWithUUID = new PersistentResource<>(childWithoutId, null, "abc", scope); + PersistentResource resourceWithIdAndUUID = new PersistentResource<>(childWithId, null, "abc", scope); assertNotEquals(resourceWithUUID, resourceWithId); assertNotEquals(resourceWithId, resourceWithUUID); @@ -2270,4 +2567,127 @@ public void testEqualsAndHashcode() { assertNotEquals(resourceWithDifferentId, resourceWithId); assertNotEquals(resourceWithId, resourceWithDifferentId); } +<<<<<<< HEAD +======= + + private PersistentResource bootstrapPersistentResource(T obj) { + return bootstrapPersistentResource(obj, mock(DataStoreTransaction.class)); + } + + private PersistentResource bootstrapPersistentResource(T obj, DataStoreTransaction tx) { + RequestScope requestScope = new RequestScope(null, null, tx, goodUser, null, elideSettings); + return new PersistentResource<>(obj, null, requestScope.getUUIDFor(obj), requestScope); + } + + private RequestScope getUserScope(User user, AuditLogger auditLogger) { + return new RequestScope(null, new JsonApiDocument(), null, user, null, + new ElideSettingsBuilder(null) + .withEntityDictionary(dictionary) + .withAuditLogger(auditLogger) + .build()); + } + + // Testing constructor, setId and non-null empty sets + private static Parent newParent(int id) { + Parent parent = new Parent(); + parent.setId(id); + parent.setChildren(new HashSet<>()); + parent.setSpouses(new HashSet<>()); + return parent; + } + + private Parent newParent(int id, Child child) { + Parent parent = new Parent(); + parent.setId(id); + parent.setChildren(Sets.newHashSet(child)); + parent.setSpouses(new HashSet<>()); + return parent; + } + + /* ChangeSpec-specific test elements */ + @Entity + @Include + @CreatePermission(expression = "allow all") + @ReadPermission(expression = "allow all") + @UpdatePermission(expression = "deny all") + @DeletePermission(expression = "allow all") + public static final class ChangeSpecModel { + @Id + public long id; + + @ReadPermission(expression = "deny all") + @UpdatePermission(expression = "deny all") + public Function checkFunction; + + @UpdatePermission(expression = "changeSpecNonCollection") + public String testAttr; + + @UpdatePermission(expression = "changeSpecCollection") + public List testColl; + + @OneToOne + @UpdatePermission(expression = "changeSpecNonCollection") + public ChangeSpecChild child; + + @ManyToMany + @UpdatePermission(expression = "changeSpecCollection") + public List otherKids; + + public ChangeSpecModel(final Function checkFunction) { + this.checkFunction = checkFunction; + } + } + + @Entity + @Include + @EqualsAndHashCode + @AllArgsConstructor + @CreatePermission(expression = "allow all") + @ReadPermission(expression = "allow all") + @UpdatePermission(expression = "allow all") + @DeletePermission(expression = "allow all") + @SharePermission + public static final class ChangeSpecChild { + @Id + public long id; + } + + public static final class ChangeSpecCollection extends OperationCheck { + @Override + public boolean ok(Object object, com.yahoo.elide.security.RequestScope requestScope, Optional changeSpec) { + if (changeSpec.isPresent() && (object instanceof ChangeSpecModel)) { + ChangeSpec spec = changeSpec.get(); + if (!(spec.getModified() instanceof Collection)) { + return false; + } + return ((ChangeSpecModel) object).checkFunction.apply(spec); + } + throw new IllegalStateException("Something is terribly wrong :("); + } + } + + public static final class ChangeSpecNonCollection extends OperationCheck { + @Override + public boolean ok(Object object, com.yahoo.elide.security.RequestScope requestScope, Optional changeSpec) { + if (changeSpec.isPresent() && (object instanceof ChangeSpecModel)) { + return ((ChangeSpecModel) object).checkFunction.apply(changeSpec.get()); + } + throw new IllegalStateException("Something is terribly wrong :("); + } + } + + public Set getRelation(PersistentResource resource, String relation) { + return resource.getRelationCheckedFiltered(getRelationship(resource.getResourceClass(), relation)); + } + + private com.yahoo.elide.request.Relationship getRelationship(Class type, String name) { + return com.yahoo.elide.request.Relationship.builder() + .name(name) + .alias(name) + .projection(EntityProjection.builder() + .type(type) + .build()) + .build(); + } +>>>>>>> ef111d6e... Create AggregationDataStore module (#845) } diff --git a/elide-core/src/test/java/com/yahoo/elide/core/TestDictionary.java b/elide-core/src/test/java/com/yahoo/elide/core/TestDictionary.java new file mode 100644 index 0000000000..245d3c7ed1 --- /dev/null +++ b/elide-core/src/test/java/com/yahoo/elide/core/TestDictionary.java @@ -0,0 +1,70 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ + +package com.yahoo.elide.core; + +import com.yahoo.elide.Injector; +import com.yahoo.elide.security.checks.Check; + +import com.google.inject.Binder; +import com.google.inject.Guice; +import com.google.inject.Module; +import com.google.inject.TypeLiteral; +import com.google.inject.name.Names; +import example.TestCheckMappings; + +import java.util.Map; +import javax.inject.Inject; +import javax.inject.Named; +import javax.inject.Singleton; + +/** + * Test Entity Dictionary. + */ +@Singleton +public class TestDictionary extends EntityDictionary { + + @Inject + public TestDictionary(Injector injector, + @Named("checkMappings") Map> checks) { + super(checks, injector); + } + + @Override + public Class lookupBoundClass(Class objClass) { + // Special handling for mocked Book class which has Entity annotation + if (objClass.getName().contains("$MockitoMock$")) { + objClass = objClass.getSuperclass(); + } + return super.lookupBoundClass(objClass); + } + + /** + * Returns a test dictionary injected with Guice. + * @return a test dictionary. + */ + public static EntityDictionary getTestDictionary() { + return getTestDictionary(TestCheckMappings.MAPPINGS); + } + + /** + * Returns a test dictionary injected with Guice. + * @param checks The security checks to setup the dictionary with. + * @return a test dictionary. + */ + public static EntityDictionary getTestDictionary(Map> checks) { + return Guice.createInjector(new Module() { + @Override + public void configure(Binder binder) { + binder.bind(Injector.class).to(TestInjector.class); + binder.bind(EntityDictionary.class).to(TestDictionary.class); + binder.bind(new TypeLiteral>>() { }) + .annotatedWith(Names.named("checkMappings")) + .toInstance(checks); + } + }).getInstance(EntityDictionary.class); + } +} diff --git a/elide-core/src/test/java/com/yahoo/elide/core/TestInjector.java b/elide-core/src/test/java/com/yahoo/elide/core/TestInjector.java new file mode 100644 index 0000000000..1d88588586 --- /dev/null +++ b/elide-core/src/test/java/com/yahoo/elide/core/TestInjector.java @@ -0,0 +1,33 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ + +package com.yahoo.elide.core; + +import com.yahoo.elide.Injector; + +import javax.inject.Inject; + +/** + * Test Dependency Injector. + */ +public class TestInjector implements Injector { + private final com.google.inject.Injector injector; + + @Inject + public TestInjector(com.google.inject.Injector injector) { + this.injector = injector; + } + + @Override + public void inject(Object entity) { + injector.injectMembers(entity); + } + + @Override + public T instantiate(Class cls) { + return injector.getInstance(cls); + } +} diff --git a/elide-core/src/test/java/com/yahoo/elide/core/TestRequestScope.java b/elide-core/src/test/java/com/yahoo/elide/core/TestRequestScope.java new file mode 100644 index 0000000000..4775a44d81 --- /dev/null +++ b/elide-core/src/test/java/com/yahoo/elide/core/TestRequestScope.java @@ -0,0 +1,53 @@ +/* + * Copyright 2016, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ + +package com.yahoo.elide.core; + +import com.yahoo.elide.ElideSettingsBuilder; +import com.yahoo.elide.jsonapi.models.JsonApiDocument; +import com.yahoo.elide.security.User; + +import java.util.Optional; +import javax.ws.rs.core.MultivaluedMap; + +/** + * Utility subclass that helps construct RequestScope objects for testing. + */ +public class TestRequestScope extends RequestScope { + + private MultivaluedMap queryParamOverrides = null; + + public TestRequestScope(DataStoreTransaction transaction, + User user, + EntityDictionary dictionary) { + super(null, new JsonApiDocument(), transaction, user, null, + new ElideSettingsBuilder(null) + .withEntityDictionary(dictionary) + .build()); + } + + public TestRequestScope(EntityDictionary dictionary, + String path, + MultivaluedMap queryParams) { + super(path, new JsonApiDocument(), null, null, queryParams, + new ElideSettingsBuilder(null) + .withEntityDictionary(dictionary) + .build()); + } + + public void setQueryParams(MultivaluedMap queryParams) { + this.queryParamOverrides = queryParams; + } + + @Override + public Optional> getQueryParams() { + if (queryParamOverrides != null) { + return Optional.of(queryParamOverrides); + } else { + return super.getQueryParams(); + } + } +} diff --git a/elide-core/src/test/java/com/yahoo/elide/core/UpdateOnCreateTest.java b/elide-core/src/test/java/com/yahoo/elide/core/UpdateOnCreateTest.java index acc34dcdcf..04ef8e8579 100644 --- a/elide-core/src/test/java/com/yahoo/elide/core/UpdateOnCreateTest.java +++ b/elide-core/src/test/java/com/yahoo/elide/core/UpdateOnCreateTest.java @@ -9,83 +9,38 @@ import static org.mockito.ArgumentMatchers.any; import static org.mockito.ArgumentMatchers.eq; import static org.mockito.Mockito.mock; +import static org.mockito.Mockito.reset; import static org.mockito.Mockito.when; import com.yahoo.elide.core.exceptions.ForbiddenAccessException; +import com.yahoo.elide.request.EntityProjection; import com.yahoo.elide.security.User; -import com.yahoo.elide.utils.coerce.CoerceUtil; + import example.Author; import example.Book; -import example.Editor; -import example.Publisher; import example.UpdateAndCreate; +import org.junit.jupiter.api.BeforeEach; import org.junit.jupiter.api.Test; -import java.io.Serializable; import java.util.Optional; public class UpdateOnCreateTest extends PersistenceResourceTestSetup { - private RequestScope userOneScope; - private RequestScope userTwoScope; - private RequestScope userThreeScope; - private RequestScope userFourScope; - - public UpdateOnCreateTest() { - super(); - init(); - } - - public void init() { - dictionary.bindEntity(Author.class); - dictionary.bindEntity(Book.class); - dictionary.bindEntity(Publisher.class); - dictionary.bindEntity(Editor.class); - dictionary.bindEntity(UpdateAndCreate.class); - - UpdateAndCreate updateAndCreateNewObject = new UpdateAndCreate(); - updateAndCreateNewObject.setId(1L); - UpdateAndCreate updateAndCreateExistingObject = new UpdateAndCreate(); - updateAndCreateExistingObject.setId(2L); - Book book = new Book(); - Author author = new Author(); - Publisher publisher = new Publisher(); - Editor editor = new Editor(); + private User userOne = new User(1); + private User userTwo = new User(2); + private User userThree = new User(3); + private User userFour = new User(4); - publisher.setEditor(editor); + private DataStoreTransaction tx = mock(DataStoreTransaction.class); - DataStoreTransaction tx = mock(DataStoreTransaction.class); - - User userOne = new User(1); - userOneScope = new RequestScope(null, null, tx, userOne, null, elideSettings); - User userTwo = new User(2); - userTwoScope = new RequestScope(null, null, tx, userTwo, null, elideSettings); - User userThree = new User(3); - userThreeScope = new RequestScope(null, null, tx, userThree, null, elideSettings); - User userFour = new User(4); - userFourScope = new RequestScope(null, null, tx, userFour, null, elideSettings); + @BeforeEach + public void beforeMethod() { + reset(tx); + } - when(tx.createNewObject(UpdateAndCreate.class)).thenReturn(updateAndCreateNewObject); - when(tx.loadObject(eq(UpdateAndCreate.class), - eq((Serializable) CoerceUtil.coerce(1, Long.class)), - eq(Optional.empty()), - any(RequestScope.class) - )).thenReturn(updateAndCreateExistingObject); - when(tx.loadObject(eq(Book.class), - eq((Serializable) CoerceUtil.coerce(1, Long.class)), - eq(Optional.empty()), - any(RequestScope.class) - )).thenReturn(book); - when(tx.loadObject(eq(Author.class), - eq((Serializable) CoerceUtil.coerce(1, Long.class)), - eq(Optional.empty()), - any(RequestScope.class) - )).thenReturn(author); - when(tx.loadObject(eq(Publisher.class), - eq((Serializable) CoerceUtil.coerce(1, Long.class)), - eq(Optional.empty()), - any(RequestScope.class) - )).thenReturn(publisher); + public UpdateOnCreateTest() { + super(); + initDictionary(); } //----------------------------------------- ** Entity Creation ** ------------------------------------------------- @@ -93,30 +48,56 @@ public void init() { //Create allowed based on class level expression @Test public void createPermissionCheckClassAnnotationForCreatingAnEntitySuccessCase() { - PersistentResource created = PersistentResource.createObject(null, UpdateAndCreate.class, userOneScope, Optional.of("1")); + RequestScope userOneScope = new TestRequestScope(tx, userOne, dictionary); + + UpdateAndCreate updateAndCreateNewObject = new UpdateAndCreate(); + when(tx.createNewObject(UpdateAndCreate.class)).thenReturn(updateAndCreateNewObject); + + PersistentResource created = PersistentResource.createObject(UpdateAndCreate.class, userOneScope, Optional.of("1")); created.getRequestScope().getPermissionExecutor().executeCommitChecks(); } //Create allowed based on field level expression @Test public void createPermissionCheckFieldAnnotationForCreatingAnEntitySuccessCase() { - PersistentResource created = PersistentResource.createObject(null, UpdateAndCreate.class, userThreeScope, Optional.of("2")); + RequestScope userThreeScope = new TestRequestScope(tx, userThree, dictionary); + + UpdateAndCreate updateAndCreateNewObject = new UpdateAndCreate(); + when(tx.createNewObject(UpdateAndCreate.class)).thenReturn(updateAndCreateNewObject); + + PersistentResource created = PersistentResource.createObject(UpdateAndCreate.class, userThreeScope, Optional.of("2")); created.getRequestScope().getPermissionExecutor().executeCommitChecks(); } //Create denied based on field level expression @Test public void createPermissionCheckFieldAnnotationForCreatingAnEntityFailureCase() { + RequestScope userFourScope = new TestRequestScope(tx, userFour, dictionary); + + UpdateAndCreate updateAndCreateNewObject = new UpdateAndCreate(); + when(tx.createNewObject(UpdateAndCreate.class)).thenReturn(updateAndCreateNewObject); assertThrows( ForbiddenAccessException.class, - () -> PersistentResource.createObject(null, UpdateAndCreate.class, userFourScope, Optional.of("3"))); + () -> PersistentResource.createObject(UpdateAndCreate.class, userFourScope, Optional.of("3"))); } //----------------------------------------- ** Update Attribute ** ------------------------------------------------ //Expression for field inherited from class level expression @Test public void updatePermissionInheritedForAttributeSuccessCase() { - PersistentResource loaded = PersistentResource.loadRecord(UpdateAndCreate.class, + RequestScope userTwoScope = new TestRequestScope(tx, userTwo, dictionary); + + UpdateAndCreate updateAndCreateExistingObject = new UpdateAndCreate(); + + when(tx.loadObject(any(), + eq(1L), + any(RequestScope.class) + )).thenReturn(updateAndCreateExistingObject); + + PersistentResource loaded = PersistentResource.loadRecord( + EntityProjection.builder() + .type(UpdateAndCreate.class) + .build(), "1", userTwoScope); loaded.updateAttribute("name", ""); @@ -125,7 +106,19 @@ public void updatePermissionInheritedForAttributeSuccessCase() { @Test public void updatePermissionInheritedForAttributeFailureCase() { - PersistentResource loaded = PersistentResource.loadRecord(UpdateAndCreate.class, + RequestScope userOneScope = new TestRequestScope(tx, userOne, dictionary); + + UpdateAndCreate updateAndCreateExistingObject = new UpdateAndCreate(); + + when(tx.loadObject(any(), + eq(1L), + any(RequestScope.class) + )).thenReturn(updateAndCreateExistingObject); + + PersistentResource loaded = PersistentResource.loadRecord( + EntityProjection.builder() + .type(UpdateAndCreate.class) + .build(), "1", userOneScope); assertThrows(ForbiddenAccessException.class, () -> loaded.updateAttribute("name", "")); @@ -134,7 +127,19 @@ public void updatePermissionInheritedForAttributeFailureCase() { //Class level expression overwritten by field level expression @Test public void updatePermissionOverwrittenForAttributeSuccessCase() { - PersistentResource loaded = PersistentResource.loadRecord(UpdateAndCreate.class, + RequestScope userFourScope = new TestRequestScope(tx, userFour, dictionary); + + UpdateAndCreate updateAndCreateExistingObject = new UpdateAndCreate(); + + when(tx.loadObject(any(), + eq(1L), + any(RequestScope.class) + )).thenReturn(updateAndCreateExistingObject); + + PersistentResource loaded = PersistentResource.loadRecord( + EntityProjection.builder() + .type(UpdateAndCreate.class) + .build(), "1", userFourScope); loaded.updateAttribute("alias", ""); @@ -143,7 +148,19 @@ public void updatePermissionOverwrittenForAttributeSuccessCase() { @Test public void updatePermissionOverwrittenForAttributeFailureCase() { - PersistentResource loaded = PersistentResource.loadRecord(UpdateAndCreate.class, + RequestScope userThreeScope = new TestRequestScope(tx, userThree, dictionary); + + UpdateAndCreate updateAndCreateExistingObject = new UpdateAndCreate(); + + when(tx.loadObject(any(), + eq(1L), + any(RequestScope.class) + )).thenReturn(updateAndCreateExistingObject); + + PersistentResource loaded = PersistentResource.loadRecord( + EntityProjection.builder() + .type(UpdateAndCreate.class) + .build(), "1", userThreeScope); assertThrows(ForbiddenAccessException.class, () -> loaded.updateAttribute("alias", "")); @@ -154,11 +171,31 @@ public void updatePermissionOverwrittenForAttributeFailureCase() { //Expression for relation inherited from class level expression @Test public void updatePermissionInheritedForRelationSuccessCase() { - PersistentResource loaded = PersistentResource.loadRecord(UpdateAndCreate.class, + RequestScope userTwoScope = new TestRequestScope(tx, userTwo, dictionary); + + UpdateAndCreate updateAndCreateExistingObject = new UpdateAndCreate(); + + when(tx.loadObject(any(), + eq(1L), + any(RequestScope.class) + )).thenReturn(updateAndCreateExistingObject); + + when(tx.loadObject(any(), + eq(2L), + any(RequestScope.class) + )).thenReturn(new Book()); + + PersistentResource loaded = PersistentResource.loadRecord( + EntityProjection.builder() + .type(UpdateAndCreate.class) + .build(), "1", userTwoScope); - PersistentResource loadedBook = PersistentResource.loadRecord(Book.class, - "1", + PersistentResource loadedBook = PersistentResource.loadRecord( + EntityProjection.builder() + .type(Book.class) + .build(), + "2", userTwoScope); loaded.addRelation("books", loadedBook); loaded.getRequestScope().getPermissionExecutor().executeCommitChecks(); @@ -166,11 +203,31 @@ public void updatePermissionInheritedForRelationSuccessCase() { @Test public void updatePermissionInheritedForRelationFailureCase() { - PersistentResource loaded = PersistentResource.loadRecord(UpdateAndCreate.class, + RequestScope userOneScope = new TestRequestScope(tx, userOne, dictionary); + + UpdateAndCreate updateAndCreateExistingObject = new UpdateAndCreate(); + + when(tx.loadObject(any(), + eq(1L), + any(RequestScope.class) + )).thenReturn(updateAndCreateExistingObject); + + when(tx.loadObject(any(), + eq(2L), + any(RequestScope.class) + )).thenReturn(new Book()); + + PersistentResource loaded = PersistentResource.loadRecord( + EntityProjection.builder() + .type(UpdateAndCreate.class) + .build(), "1", userOneScope); - PersistentResource loadedBook = PersistentResource.loadRecord(Book.class, - "1", + PersistentResource loadedBook = PersistentResource.loadRecord( + EntityProjection.builder() + .type(Book.class) + .build(), + "2", userOneScope); assertThrows(ForbiddenAccessException.class, () -> loaded.addRelation("books", loadedBook)); } @@ -178,11 +235,33 @@ public void updatePermissionInheritedForRelationFailureCase() { //Class level expression overwritten by field level expression @Test public void updatePermissionOverwrittenForRelationSuccessCase() { - PersistentResource loaded = PersistentResource.loadRecord(UpdateAndCreate.class, + RequestScope userThreeScope = new TestRequestScope(tx, new User(3), dictionary); + + + UpdateAndCreate updateAndCreateExistingObject = new UpdateAndCreate(); + updateAndCreateExistingObject.setId(1L); + + when(tx.loadObject(any(), + eq(1L), + any(RequestScope.class) + )).thenReturn(updateAndCreateExistingObject); + + when(tx.loadObject(any(), + eq(2L), + any(RequestScope.class) + )).thenReturn(new Author()); + + PersistentResource loaded = PersistentResource.loadRecord( + EntityProjection.builder() + .type(UpdateAndCreate.class) + .build(), "1", userThreeScope); - PersistentResource loadedAuthor = PersistentResource.loadRecord(Author.class, - "1", + PersistentResource loadedAuthor = PersistentResource.loadRecord( + EntityProjection.builder() + .type(Author.class) + .build(), + "2", userThreeScope); loaded.addRelation("author", loadedAuthor); loaded.getRequestScope().getPermissionExecutor().executeCommitChecks(); @@ -190,11 +269,32 @@ public void updatePermissionOverwrittenForRelationSuccessCase() { @Test public void updatePermissionOverwrittenForRelationFailureCase() { - PersistentResource loaded = PersistentResource.loadRecord(UpdateAndCreate.class, + RequestScope userTwoScope = new TestRequestScope(tx, userTwo, dictionary); + + UpdateAndCreate updateAndCreateExistingObject = new UpdateAndCreate(); + + when(tx.loadObject(any(), + eq(1L), + any(RequestScope.class) + )).thenReturn(updateAndCreateExistingObject); + + when(tx.loadObject(any(), + eq(2L), + any(RequestScope.class) + )).thenReturn(new Author()); + + PersistentResource loaded = PersistentResource.loadRecord( + EntityProjection.builder() + .type(UpdateAndCreate.class) + + .build(), "1", userTwoScope); - PersistentResource loadedAuthor = PersistentResource.loadRecord(Author.class, - "1", + PersistentResource loadedAuthor = PersistentResource.loadRecord( + EntityProjection.builder() + .type(Author.class) + .build(), + "2", userTwoScope); assertThrows(ForbiddenAccessException.class, () -> loaded.addRelation("author", loadedAuthor)); } @@ -203,54 +303,99 @@ public void updatePermissionOverwrittenForRelationFailureCase() { //Expression for field inherited from class level expression @Test public void createPermissionInheritedForAttributeSuccessCase() { - PersistentResource created = PersistentResource.createObject(null, UpdateAndCreate.class, userOneScope, Optional.of("4")); + RequestScope userOneScope = new TestRequestScope(tx, userOne, dictionary); + + UpdateAndCreate updateAndCreateNewObject = new UpdateAndCreate(); + when(tx.createNewObject(UpdateAndCreate.class)).thenReturn(updateAndCreateNewObject); + + PersistentResource created = PersistentResource.createObject(UpdateAndCreate.class, userOneScope, Optional.of("4")); created.updateAttribute("name", ""); created.getRequestScope().getPermissionExecutor().executeCommitChecks(); } @Test public void createPermissionInheritedForAttributeFailureCase() { - PersistentResource created = PersistentResource.createObject(null, UpdateAndCreate.class, userThreeScope, Optional.of("5")); + RequestScope userThreeScope = new TestRequestScope(tx, userThree, dictionary); + + UpdateAndCreate updateAndCreateNewObject = new UpdateAndCreate(); + when(tx.createNewObject(UpdateAndCreate.class)).thenReturn(updateAndCreateNewObject); + + PersistentResource created = PersistentResource.createObject(UpdateAndCreate.class, userThreeScope, Optional.of("5")); assertThrows(ForbiddenAccessException.class, () -> created.updateAttribute("name", "")); } //Class level expression overwritten by field level expression @Test public void createPermissionOverwrittenForAttributeSuccessCase() { - PersistentResource created = PersistentResource.createObject(null, UpdateAndCreate.class, userThreeScope, Optional.of("6")); + RequestScope userThreeScope = new TestRequestScope(tx, userThree, dictionary); + + UpdateAndCreate updateAndCreateNewObject = new UpdateAndCreate(); + when(tx.createNewObject(UpdateAndCreate.class)).thenReturn(updateAndCreateNewObject); + + PersistentResource created = PersistentResource.createObject(UpdateAndCreate.class, userThreeScope, Optional.of("6")); created.updateAttribute("alias", ""); created.getRequestScope().getPermissionExecutor().executeCommitChecks(); } @Test public void createPermissionOverwrittenForAttributeFailureCase() { + RequestScope userFourScope = new TestRequestScope(tx, userFour, dictionary); + + UpdateAndCreate updateAndCreateNewObject = new UpdateAndCreate(); + when(tx.createNewObject(UpdateAndCreate.class)).thenReturn(updateAndCreateNewObject); assertThrows( ForbiddenAccessException.class, () -> { PersistentResource created = - PersistentResource.createObject(null, UpdateAndCreate.class, userFourScope, Optional.of("7")); + PersistentResource.createObject(UpdateAndCreate.class, userFourScope, Optional.of("7")); created.updateAttribute("alias", ""); } ); } - //----------------------------------------- ** Update Relation On Create ** -------------------------------------- //Expression for relation inherited from class level expression @Test public void createPermissionInheritedForRelationSuccessCase() { - PersistentResource created = PersistentResource.createObject(null, UpdateAndCreate.class, userOneScope, Optional.of("8")); - PersistentResource loadedBook = PersistentResource.loadRecord(Book.class, - "1", + RequestScope userOneScope = new TestRequestScope(tx, userOne, dictionary); + + UpdateAndCreate updateAndCreateNewObject = new UpdateAndCreate(); + when(tx.createNewObject(UpdateAndCreate.class)).thenReturn(updateAndCreateNewObject); + + when(tx.loadObject(any(), + eq(2L), + any(RequestScope.class) + )).thenReturn(new Book()); + + PersistentResource created = PersistentResource.createObject(UpdateAndCreate.class, userOneScope, Optional.of("8")); + PersistentResource loadedBook = PersistentResource.loadRecord( + EntityProjection.builder() + .type(Book.class) + .build(), + "2", userOneScope); + created.addRelation("books", loadedBook); created.getRequestScope().getPermissionExecutor().executeCommitChecks(); } @Test public void createPermissionInheritedForRelationFailureCase() { - PersistentResource created = PersistentResource.createObject(null, UpdateAndCreate.class, userThreeScope, Optional.of("9")); - PersistentResource loadedBook = PersistentResource.loadRecord(Book.class, - "1", + RequestScope userThreeScope = new TestRequestScope(tx, userThree, dictionary); + + UpdateAndCreate updateAndCreateNewObject = new UpdateAndCreate(); + when(tx.createNewObject(UpdateAndCreate.class)).thenReturn(updateAndCreateNewObject); + + when(tx.loadObject(any(), + eq(2L), + any(RequestScope.class) + )).thenReturn(new Book()); + + PersistentResource created = PersistentResource.createObject(UpdateAndCreate.class, userThreeScope, Optional.of("9")); + PersistentResource loadedBook = PersistentResource.loadRecord( + EntityProjection.builder() + .type(Book.class) + .build(), + "2", userThreeScope); assertThrows(ForbiddenAccessException.class, () -> created.addRelation("books", loadedBook)); } @@ -258,9 +403,22 @@ public void createPermissionInheritedForRelationFailureCase() { //Class level expression overwritten by field level expression @Test public void createPermissionOverwrittenForRelationSuccessCase() { - PersistentResource created = PersistentResource.createObject(null, UpdateAndCreate.class, userTwoScope, Optional.of("10")); - PersistentResource loadedAuthor = PersistentResource.loadRecord(Author.class, - "1", + RequestScope userTwoScope = new TestRequestScope(tx, userTwo, dictionary); + + UpdateAndCreate updateAndCreateNewObject = new UpdateAndCreate(); + when(tx.createNewObject(UpdateAndCreate.class)).thenReturn(updateAndCreateNewObject); + + when(tx.loadObject(any(), + eq(2L), + any(RequestScope.class) + )).thenReturn(new Author()); + + PersistentResource created = PersistentResource.createObject(UpdateAndCreate.class, userTwoScope, Optional.of("10")); + PersistentResource loadedAuthor = PersistentResource.loadRecord( + EntityProjection.builder() + .type(Author.class) + .build(), + "2", userTwoScope); created.addRelation("author", loadedAuthor); created.getRequestScope().getPermissionExecutor().executeCommitChecks(); @@ -268,9 +426,22 @@ public void createPermissionOverwrittenForRelationSuccessCase() { @Test public void createPermissionOverwrittenForRelationFailureCase() { - PersistentResource created = PersistentResource.createObject(null, UpdateAndCreate.class, userOneScope, Optional.of("11")); - PersistentResource loadedAuthor = PersistentResource.loadRecord(Author.class, - "1", + RequestScope userOneScope = new TestRequestScope(tx, userOne, dictionary); + + UpdateAndCreate updateAndCreateNewObject = new UpdateAndCreate(); + when(tx.createNewObject(UpdateAndCreate.class)).thenReturn(updateAndCreateNewObject); + + when(tx.loadObject(any(), + eq(2L), + any(RequestScope.class) + )).thenReturn(new Author()); + + PersistentResource created = PersistentResource.createObject(UpdateAndCreate.class, userOneScope, Optional.of("11")); + PersistentResource loadedAuthor = PersistentResource.loadRecord( + EntityProjection.builder() + .type(Author.class) + .build(), + "2", userOneScope); assertThrows(ForbiddenAccessException.class, () -> created.addRelation("author", loadedAuthor)); } diff --git a/elide-core/src/test/java/com/yahoo/elide/core/datastore/inmemory/InMemoryStoreTransactionTest.java b/elide-core/src/test/java/com/yahoo/elide/core/datastore/inmemory/InMemoryStoreTransactionTest.java index bf09246bb6..ba2b8b4b61 100644 --- a/elide-core/src/test/java/com/yahoo/elide/core/datastore/inmemory/InMemoryStoreTransactionTest.java +++ b/elide-core/src/test/java/com/yahoo/elide/core/datastore/inmemory/InMemoryStoreTransactionTest.java @@ -6,6 +6,7 @@ package com.yahoo.elide.core.datastore.inmemory; import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.junit.jupiter.api.Assertions.assertNull; import static org.junit.jupiter.api.Assertions.assertTrue; import static org.mockito.ArgumentMatchers.any; import static org.mockito.ArgumentMatchers.eq; @@ -27,15 +28,19 @@ import com.yahoo.elide.core.filter.expression.FilterExpression; import com.yahoo.elide.core.pagination.Pagination; import com.yahoo.elide.core.sort.Sorting; +import com.yahoo.elide.request.EntityProjection; +import com.yahoo.elide.request.Relationship; import com.google.common.collect.Lists; import com.google.common.collect.Sets; + import example.Author; import example.Book; import example.Editor; import example.Publisher; import org.junit.jupiter.api.BeforeEach; import org.junit.jupiter.api.Test; +import org.mockito.ArgumentCaptor; import java.util.ArrayList; import java.util.Collection; @@ -43,7 +48,6 @@ import java.util.HashSet; import java.util.List; import java.util.Map; -import java.util.Optional; import java.util.Set; import java.util.stream.Collectors; @@ -53,7 +57,7 @@ public class InMemoryStoreTransactionTest { private RequestScope scope = mock(RequestScope.class); private InMemoryStoreTransaction inMemoryStoreTransaction = new InMemoryStoreTransaction(wrappedTransaction); private EntityDictionary dictionary; - private Set books = new HashSet<>(); + private Set books = new HashSet<>(); private Book book1; private Book book2; private Book book3; @@ -128,24 +132,17 @@ public void testFullFilterPredicatePushDown() { FilterExpression expression = new InPredicate(new Path(Book.class, dictionary, "genre"), "Literary Fiction"); - when(wrappedTransaction.supportsFiltering(eq(Book.class), - any())).thenReturn(DataStoreTransaction.FeatureSupport.FULL); - when(wrappedTransaction.loadObjects(eq(Book.class), eq(Optional.of(expression)), - eq(Optional.empty()), eq(Optional.empty()), eq(scope))).thenReturn((Set) books); + EntityProjection projection = EntityProjection.builder() + .type(Book.class) + .filterExpression(expression) + .build(); - Collection loaded = (Collection) inMemoryStoreTransaction.loadObjects( - Book.class, - Optional.of(expression), - Optional.empty(), - Optional.empty(), - scope); + when(wrappedTransaction.supportsFiltering(eq(Book.class), any())).thenReturn(DataStoreTransaction.FeatureSupport.FULL); + when(wrappedTransaction.loadObjects(eq(projection), eq(scope))).thenReturn(books); - verify(wrappedTransaction, times(1)).loadObjects( - eq(Book.class), - eq(Optional.of(expression)), - eq(Optional.empty()), - eq(Optional.empty()), - eq(scope)); + Collection loaded = (Collection) inMemoryStoreTransaction.loadObjects(projection, scope); + + verify(wrappedTransaction, times(1)).loadObjects(eq(projection), eq(scope)); assertEquals(3, loaded.size()); assertTrue(loaded.contains(book1)); @@ -158,89 +155,66 @@ public void testTransactionRequiresInMemoryFilterDuringGetRelation() { FilterExpression expression = new InPredicate(new Path(Book.class, dictionary, "genre"), "Literary Fiction"); + Relationship relationship = Relationship.builder() + .projection(EntityProjection.builder() + .type(Book.class) + .filterExpression(expression) + .build()) + .name("books") + .alias("books") + .build(); + + ArgumentCaptor relationshipArgument = ArgumentCaptor.forClass(Relationship.class); + when(scope.getNewPersistentResources()).thenReturn(Sets.newHashSet(mock(PersistentResource.class))); when(wrappedTransaction.supportsFiltering(eq(Book.class), any())).thenReturn(DataStoreTransaction.FeatureSupport.FULL); - when(wrappedTransaction.getRelation(eq(inMemoryStoreTransaction), eq(author), eq("books"), - eq(Optional.empty()), eq(Optional.empty()), eq(Optional.empty()), eq(scope))).thenReturn(books); + when(wrappedTransaction.getRelation(eq(inMemoryStoreTransaction), eq(author), any(), eq(scope))).thenReturn(books); Collection loaded = (Collection) inMemoryStoreTransaction.getRelation( - inMemoryStoreTransaction, - author, - "books", - Optional.of(expression), - Optional.empty(), - Optional.empty(), - scope); + inMemoryStoreTransaction, author, relationship, scope); verify(wrappedTransaction, times(1)).getRelation( eq(inMemoryStoreTransaction), eq(author), - eq("books"), - eq(Optional.empty()), - eq(Optional.empty()), - eq(Optional.empty()), + relationshipArgument.capture(), eq(scope)); + assertNull(relationshipArgument.getValue().getProjection().getFilterExpression()); + assertNull(relationshipArgument.getValue().getProjection().getSorting()); + assertNull(relationshipArgument.getValue().getProjection().getPagination()); + assertEquals(2, loaded.size()); assertTrue(loaded.contains(book1)); assertTrue(loaded.contains(book3)); } @Test - public void testTransactionRequiresInMemoryFilterDuringLoad() { + public void testDataStoreRequiresTotalInMemoryFilter() { FilterExpression expression = new InPredicate(new Path(Book.class, dictionary, "genre"), "Literary Fiction"); - when(wrappedTransaction.supportsFiltering(eq(Book.class), - any())).thenReturn(DataStoreTransaction.FeatureSupport.FULL); - when(wrappedTransaction.loadObjects(eq(Book.class), eq(Optional.of(expression)), - eq(Optional.empty()), eq(Optional.empty()), eq(scope))).thenReturn((Set) books); + EntityProjection projection = EntityProjection.builder() + .type(Book.class) + .filterExpression(expression) + .build(); - Collection loaded = (Collection) inMemoryStoreTransaction.loadObjects( - Book.class, - Optional.of(expression), - Optional.empty(), - Optional.empty(), - scope); - - verify(wrappedTransaction, times(1)).loadObjects( - eq(Book.class), - eq(Optional.of(expression)), - eq(Optional.empty()), - eq(Optional.empty()), - eq(scope)); - - assertEquals(3, loaded.size()); - assertTrue(loaded.contains(book1)); - assertTrue(loaded.contains(book2)); - assertTrue(loaded.contains(book3)); - } - - @Test - public void testDataStoreRequiresTotalInMemoryFilter() { - FilterExpression expression = - new InPredicate(new Path(Book.class, dictionary, "genre"), "Literary Fiction"); + ArgumentCaptor projectionArgument = ArgumentCaptor.forClass(EntityProjection.class); when(wrappedTransaction.supportsFiltering(eq(Book.class), any())).thenReturn(DataStoreTransaction.FeatureSupport.NONE); - when(wrappedTransaction.loadObjects(eq(Book.class), eq(Optional.empty()), - eq(Optional.empty()), eq(Optional.empty()), eq(scope))).thenReturn((Set) books); - Collection loaded = (Collection) inMemoryStoreTransaction.loadObjects( - Book.class, - Optional.of(expression), - Optional.empty(), - Optional.empty(), - scope); + when(wrappedTransaction.loadObjects(any(), eq(scope))).thenReturn(books); + + Collection loaded = (Collection) inMemoryStoreTransaction.loadObjects(projection, scope); verify(wrappedTransaction, times(1)).loadObjects( - eq(Book.class), - eq(Optional.empty()), - eq(Optional.empty()), - eq(Optional.empty()), + projectionArgument.capture(), eq(scope)); + assertNull(projectionArgument.getValue().getFilterExpression()); + assertNull(projectionArgument.getValue().getPagination()); + assertNull(projectionArgument.getValue().getSorting()); assertEquals(2, loaded.size()); assertTrue(loaded.contains(book1)); assertTrue(loaded.contains(book3)); @@ -254,25 +228,28 @@ public void testDataStoreRequiresPartialInMemoryFilter() { new InPredicate(new Path(Book.class, dictionary, "editor.firstName"), "Jane"); FilterExpression expression = new AndFilterExpression(expression1, expression2); + EntityProjection projection = EntityProjection.builder() + .type(Book.class) + .filterExpression(expression) + .build(); + + ArgumentCaptor projectionArgument = ArgumentCaptor.forClass(EntityProjection.class); + when(wrappedTransaction.supportsFiltering(eq(Book.class), any())).thenReturn(DataStoreTransaction.FeatureSupport.PARTIAL); - when(wrappedTransaction.loadObjects(eq(Book.class), eq(Optional.of(expression1)), - eq(Optional.empty()), eq(Optional.empty()), eq(scope))).thenReturn((Set) books); + when(wrappedTransaction.loadObjects(any(), eq(scope))).thenReturn(books); Collection loaded = (Collection) inMemoryStoreTransaction.loadObjects( - Book.class, - Optional.of(expression), - Optional.empty(), - Optional.empty(), + projection, scope); verify(wrappedTransaction, times(1)).loadObjects( - eq(Book.class), - eq(Optional.of(expression1)), - eq(Optional.empty()), - eq(Optional.empty()), + projectionArgument.capture(), eq(scope)); + assertEquals(projectionArgument.getValue().getFilterExpression(), expression1); + assertNull(projectionArgument.getValue().getPagination()); + assertNull(projectionArgument.getValue().getSorting()); assertEquals(1, loaded.size()); assertTrue(loaded.contains(book3)); } @@ -284,28 +261,30 @@ public void testSortingPushDown() { Sorting sorting = new Sorting(sortOrder); + EntityProjection projection = EntityProjection.builder() + .type(Book.class) + .sorting(sorting) + .build(); + + ArgumentCaptor projectionArgument = ArgumentCaptor.forClass(EntityProjection.class); + when(wrappedTransaction.supportsFiltering(eq(Book.class), any())).thenReturn(DataStoreTransaction.FeatureSupport.FULL); when(wrappedTransaction.supportsSorting(eq(Book.class), any())).thenReturn(true); - - when(wrappedTransaction.loadObjects(eq(Book.class), eq(Optional.empty()), - eq(Optional.of(sorting)), eq(Optional.empty()), eq(scope))).thenReturn((Set) books); + when(wrappedTransaction.loadObjects(any(), eq(scope))).thenReturn(books); Collection loaded = (Collection) inMemoryStoreTransaction.loadObjects( - Book.class, - Optional.empty(), - Optional.of(sorting), - Optional.empty(), + projection, scope); verify(wrappedTransaction, times(1)).loadObjects( - eq(Book.class), - eq(Optional.empty()), - eq(Optional.of(sorting)), - eq(Optional.empty()), + projectionArgument.capture(), eq(scope)); + assertNull(projectionArgument.getValue().getFilterExpression()); + assertNull(projectionArgument.getValue().getPagination()); + assertEquals(projectionArgument.getValue().getSorting(), sorting); assertEquals(3, loaded.size()); } @@ -316,28 +295,30 @@ public void testDataStoreRequiresInMemorySorting() { Sorting sorting = new Sorting(sortOrder); + EntityProjection projection = EntityProjection.builder() + .type(Book.class) + .sorting(sorting) + .build(); + + ArgumentCaptor projectionArgument = ArgumentCaptor.forClass(EntityProjection.class); + when(wrappedTransaction.supportsFiltering(eq(Book.class), any())).thenReturn(DataStoreTransaction.FeatureSupport.FULL); when(wrappedTransaction.supportsSorting(eq(Book.class), any())).thenReturn(false); - - when(wrappedTransaction.loadObjects(eq(Book.class), eq(Optional.empty()), - eq(Optional.empty()), eq(Optional.empty()), eq(scope))).thenReturn((Set) books); + when(wrappedTransaction.loadObjects(any(), eq(scope))).thenReturn(books); Collection loaded = (Collection) inMemoryStoreTransaction.loadObjects( - Book.class, - Optional.empty(), - Optional.of(sorting), - Optional.empty(), + projection, scope); verify(wrappedTransaction, times(1)).loadObjects( - eq(Book.class), - eq(Optional.empty()), - eq(Optional.empty()), - eq(Optional.empty()), + projectionArgument.capture(), eq(scope)); + assertNull(projectionArgument.getValue().getFilterExpression()); + assertNull(projectionArgument.getValue().getPagination()); + assertNull(projectionArgument.getValue().getSorting()); assertEquals(3, loaded.size()); List bookTitles = loaded.stream().map((o) -> ((Book) o).getTitle()).collect(Collectors.toList()); @@ -354,28 +335,31 @@ public void testFilteringRequiresInMemorySorting() { Sorting sorting = new Sorting(sortOrder); + EntityProjection projection = EntityProjection.builder() + .type(Book.class) + .filterExpression(expression) + .sorting(sorting) + .build(); + + ArgumentCaptor projectionArgument = ArgumentCaptor.forClass(EntityProjection.class); + when(wrappedTransaction.supportsFiltering(eq(Book.class), any())).thenReturn(DataStoreTransaction.FeatureSupport.NONE); when(wrappedTransaction.supportsSorting(eq(Book.class), any())).thenReturn(true); - - when(wrappedTransaction.loadObjects(eq(Book.class), eq(Optional.empty()), - eq(Optional.empty()), eq(Optional.empty()), eq(scope))).thenReturn((Set) books); + when(wrappedTransaction.loadObjects(any(), eq(scope))).thenReturn(books); Collection loaded = (Collection) inMemoryStoreTransaction.loadObjects( - Book.class, - Optional.of(expression), - Optional.of(sorting), - Optional.empty(), + projection, scope); verify(wrappedTransaction, times(1)).loadObjects( - eq(Book.class), - eq(Optional.empty()), - eq(Optional.empty()), - eq(Optional.empty()), + projectionArgument.capture(), eq(scope)); + assertNull(projectionArgument.getValue().getFilterExpression()); + assertNull(projectionArgument.getValue().getPagination()); + assertNull(projectionArgument.getValue().getSorting()); assertEquals(2, loaded.size()); List bookTitles = loaded.stream().map((o) -> ((Book) o).getTitle()).collect(Collectors.toList()); @@ -386,27 +370,30 @@ public void testFilteringRequiresInMemorySorting() { public void testPaginationPushDown() { Pagination pagination = Pagination.getDefaultPagination(elideSettings); + EntityProjection projection = EntityProjection.builder() + .type(Book.class) + .pagination(pagination) + .build(); + + ArgumentCaptor projectionArgument = ArgumentCaptor.forClass(EntityProjection.class); + when(wrappedTransaction.supportsFiltering(eq(Book.class), any())).thenReturn(DataStoreTransaction.FeatureSupport.FULL); when(wrappedTransaction.supportsPagination(eq(Book.class))).thenReturn(true); - when(wrappedTransaction.loadObjects(eq(Book.class), eq(Optional.empty()), - eq(Optional.empty()), eq(Optional.of(pagination)), eq(scope))).thenReturn((Set) books); + when(wrappedTransaction.loadObjects(any(), eq(scope))).thenReturn(books); Collection loaded = (Collection) inMemoryStoreTransaction.loadObjects( - Book.class, - Optional.empty(), - Optional.empty(), - Optional.of(pagination), + projection, scope); verify(wrappedTransaction, times(1)).loadObjects( - eq(Book.class), - eq(Optional.empty()), - eq(Optional.empty()), - eq(Optional.of(pagination)), + projectionArgument.capture(), eq(scope)); + assertNull(projectionArgument.getValue().getFilterExpression()); + assertEquals(projectionArgument.getValue().getPagination(), pagination); + assertNull(projectionArgument.getValue().getSorting()); assertEquals(3, loaded.size()); } @@ -414,27 +401,30 @@ public void testPaginationPushDown() { public void testDataStoreRequiresInMemoryPagination() { Pagination pagination = Pagination.getDefaultPagination(elideSettings); + EntityProjection projection = EntityProjection.builder() + .type(Book.class) + .pagination(pagination) + .build(); + + ArgumentCaptor projectionArgument = ArgumentCaptor.forClass(EntityProjection.class); + when(wrappedTransaction.supportsFiltering(eq(Book.class), any())).thenReturn(DataStoreTransaction.FeatureSupport.FULL); when(wrappedTransaction.supportsPagination(eq(Book.class))).thenReturn(false); - when(wrappedTransaction.loadObjects(eq(Book.class), eq(Optional.empty()), - eq(Optional.empty()), eq(Optional.empty()), eq(scope))).thenReturn((Set) books); + when(wrappedTransaction.loadObjects(any(), eq(scope))).thenReturn(books); Collection loaded = (Collection) inMemoryStoreTransaction.loadObjects( - Book.class, - Optional.empty(), - Optional.empty(), - Optional.of(pagination), + projection, scope); verify(wrappedTransaction, times(1)).loadObjects( - eq(Book.class), - eq(Optional.empty()), - eq(Optional.empty()), - eq(Optional.empty()), + projectionArgument.capture(), eq(scope)); + assertNull(projectionArgument.getValue().getFilterExpression()); + assertNull(projectionArgument.getValue().getPagination()); + assertNull(projectionArgument.getValue().getSorting()); assertEquals(3, loaded.size()); assertTrue(loaded.contains(book1)); assertTrue(loaded.contains(book2)); @@ -448,27 +438,31 @@ public void testFilteringRequiresInMemoryPagination() { Pagination pagination = Pagination.getDefaultPagination(elideSettings); + EntityProjection projection = EntityProjection.builder() + .type(Book.class) + .filterExpression(expression) + .pagination(pagination) + .build(); + + ArgumentCaptor projectionArgument = ArgumentCaptor.forClass(EntityProjection.class); + when(wrappedTransaction.supportsFiltering(eq(Book.class), any())).thenReturn(DataStoreTransaction.FeatureSupport.NONE); when(wrappedTransaction.supportsPagination(eq(Book.class))).thenReturn(true); - when(wrappedTransaction.loadObjects(eq(Book.class), eq(Optional.empty()), - eq(Optional.empty()), eq(Optional.empty()), eq(scope))).thenReturn((Set) books); + when(wrappedTransaction.loadObjects(any(), eq(scope))).thenReturn(books); Collection loaded = (Collection) inMemoryStoreTransaction.loadObjects( - Book.class, - Optional.of(expression), - Optional.empty(), - Optional.of(pagination), + projection, scope); verify(wrappedTransaction, times(1)).loadObjects( - eq(Book.class), - eq(Optional.empty()), - eq(Optional.empty()), - eq(Optional.empty()), + projectionArgument.capture(), eq(scope)); + assertNull(projectionArgument.getValue().getFilterExpression()); + assertNull(projectionArgument.getValue().getPagination()); + assertNull(projectionArgument.getValue().getSorting()); assertEquals(2, loaded.size()); assertTrue(loaded.contains(book1)); assertTrue(loaded.contains(book3)); @@ -483,29 +477,33 @@ public void testSortingRequiresInMemoryPagination() { Sorting sorting = new Sorting(sortOrder); + EntityProjection projection = EntityProjection.builder() + .type(Book.class) + .sorting(sorting) + .pagination(pagination) + .build(); + + ArgumentCaptor projectionArgument = ArgumentCaptor.forClass(EntityProjection.class); + when(wrappedTransaction.supportsFiltering(eq(Book.class), any())).thenReturn(DataStoreTransaction.FeatureSupport.FULL); when(wrappedTransaction.supportsSorting(eq(Book.class), any())).thenReturn(false); when(wrappedTransaction.supportsPagination(eq(Book.class))).thenReturn(true); - when(wrappedTransaction.loadObjects(eq(Book.class), eq(Optional.empty()), - eq(Optional.empty()), eq(Optional.empty()), eq(scope))).thenReturn((Set) books); + when(wrappedTransaction.loadObjects(any(), eq(scope))).thenReturn(books); Collection loaded = (Collection) inMemoryStoreTransaction.loadObjects( - Book.class, - Optional.empty(), - Optional.of(sorting), - Optional.of(pagination), + projection, scope); verify(wrappedTransaction, times(1)).loadObjects( - eq(Book.class), - eq(Optional.empty()), - eq(Optional.empty()), - eq(Optional.empty()), + projectionArgument.capture(), eq(scope)); + assertNull(projectionArgument.getValue().getFilterExpression()); + assertNull(projectionArgument.getValue().getPagination()); + assertNull(projectionArgument.getValue().getSorting()); assertEquals(3, loaded.size()); assertTrue(loaded.contains(book1)); assertTrue(loaded.contains(book2)); diff --git a/elide-core/src/test/java/com/yahoo/elide/core/datastore/wrapped/TransactionWrapperTest.java b/elide-core/src/test/java/com/yahoo/elide/core/datastore/wrapped/TransactionWrapperTest.java index 4a84d6a615..0141bffefb 100644 --- a/elide-core/src/test/java/com/yahoo/elide/core/datastore/wrapped/TransactionWrapperTest.java +++ b/elide-core/src/test/java/com/yahoo/elide/core/datastore/wrapped/TransactionWrapperTest.java @@ -10,21 +10,20 @@ import static org.junit.jupiter.api.Assertions.assertTrue; import static org.mockito.ArgumentMatchers.any; import static org.mockito.ArgumentMatchers.eq; +import static org.mockito.ArgumentMatchers.isA; import static org.mockito.Mockito.mock; import static org.mockito.Mockito.times; import static org.mockito.Mockito.verify; import static org.mockito.Mockito.when; import com.yahoo.elide.core.DataStoreTransaction; +import com.yahoo.elide.request.Attribute; import com.yahoo.elide.security.User; import org.junit.jupiter.api.Test; -import java.util.Optional; - public class TransactionWrapperTest { private class TestTransactionWrapper extends TransactionWrapper { - public TestTransactionWrapper(DataStoreTransaction wrapped) { super(wrapped); } @@ -84,12 +83,11 @@ public void testLoadObjects() throws Exception { DataStoreTransaction wrapper = new TestTransactionWrapper(wrapped); Iterable expected = mock(Iterable.class); - when(wrapped.loadObjects(any(), any(), any(), any(), any())).thenReturn(expected); + when(wrapped.loadObjects(any(), any())).thenReturn(expected); - Iterable actual = wrapper.loadObjects(null, Optional.empty(), - Optional.empty(), Optional.empty(), null); + Iterable actual = wrapper.loadObjects(null, null); - verify(wrapped, times(1)).loadObjects(any(), any(), any(), any(), any()); + verify(wrapped, times(1)).loadObjects(any(), any()); assertEquals(expected, actual); } @@ -184,11 +182,11 @@ public void testGetAttribute() { DataStoreTransaction wrapped = mock(DataStoreTransaction.class); DataStoreTransaction wrapper = new TestTransactionWrapper(wrapped); - when(wrapped.getAttribute(any(), any(), any())).thenReturn(1L); + when(wrapped.getAttribute(any(), isA(Attribute.class), any())).thenReturn(1L); - Object actual = wrapper.getAttribute(null, null, null); + Object actual = wrapper.getAttribute(null, Attribute.builder().name("foo").type(String.class).build(), null); - verify(wrapped, times(1)).getAttribute(any(), any(), any()); + verify(wrapped, times(1)).getAttribute(any(), isA(Attribute.class), any()); assertEquals(1L, actual); } @@ -197,9 +195,9 @@ public void testSetAttribute() { DataStoreTransaction wrapped = mock(DataStoreTransaction.class); DataStoreTransaction wrapper = new TestTransactionWrapper(wrapped); - wrapper.setAttribute(null, null, null, null); + wrapper.setAttribute(null, null, null); - verify(wrapped, times(1)).setAttribute(any(), any(), any(), any()); + verify(wrapped, times(1)).setAttribute(any(), any(), any()); } @Test @@ -227,12 +225,11 @@ public void testGetRelation() { DataStoreTransaction wrapped = mock(DataStoreTransaction.class); DataStoreTransaction wrapper = new TestTransactionWrapper(wrapped); - when(wrapped.getRelation(any(), any(), any(), any(), any(), any(), any())).thenReturn(1L); + when(wrapped.getRelation(any(), any(), any(), any())).thenReturn(1L); - Object actual = wrapper.getRelation(null, null, null, null, - null, null, null); + Object actual = wrapper.getRelation(null, null, null, null); - verify(wrapped, times(1)).getRelation(any(), any(), any(), any(), any(), any(), any()); + verify(wrapped, times(1)).getRelation(any(), any(), any(), any()); assertEquals(1L, actual); } @@ -241,11 +238,11 @@ public void testLoadObject() { DataStoreTransaction wrapped = mock(DataStoreTransaction.class); DataStoreTransaction wrapper = new TestTransactionWrapper(wrapped); - when(wrapped.loadObject(any(), any(), any(), any())).thenReturn(1L); + when(wrapped.loadObject(any(), any(), any())).thenReturn(1L); - Object actual = wrapper.loadObject(null, null, null, null); + Object actual = wrapper.loadObject(null, null, null); - verify(wrapped, times(1)).loadObject(any(), any(), any(), any()); + verify(wrapped, times(1)).loadObject(any(), any(), any()); assertEquals(1L, actual); } } diff --git a/elide-core/src/test/java/com/yahoo/elide/core/utils/ClassScannerTest.java b/elide-core/src/test/java/com/yahoo/elide/core/utils/ClassScannerTest.java index 5b6eb5a866..b39788aa48 100644 --- a/elide-core/src/test/java/com/yahoo/elide/core/utils/ClassScannerTest.java +++ b/elide-core/src/test/java/com/yahoo/elide/core/utils/ClassScannerTest.java @@ -9,6 +9,7 @@ import static org.junit.jupiter.api.Assertions.assertTrue; import com.yahoo.elide.annotation.ReadPermission; +import com.yahoo.elide.annotation.UpdatePermission; import com.yahoo.elide.utils.ClassScanner; import org.apache.commons.collections4.IterableUtils; @@ -37,4 +38,14 @@ public void testGetAllAnnotatedClasses() { assertEquals(12, classes.size(), "Actual: " + classes); classes.forEach(cls -> assertTrue(cls.isAnnotationPresent(ReadPermission.class))); } + + @Test + public void testGetAnyAnnotatedClasses() { + Set> classes = ClassScanner.getAnnotatedClasses(ReadPermission.class, UpdatePermission.class); + assertEquals(37, classes.size()); + for (Class cls : classes) { + assertTrue(cls.isAnnotationPresent(ReadPermission.class) + || cls.isAnnotationPresent(UpdatePermission.class)); + } + } } diff --git a/elide-core/src/test/java/com/yahoo/elide/jsonapi/EntityProjectionMakerTest.java b/elide-core/src/test/java/com/yahoo/elide/jsonapi/EntityProjectionMakerTest.java new file mode 100644 index 0000000000..e970bb0233 --- /dev/null +++ b/elide-core/src/test/java/com/yahoo/elide/jsonapi/EntityProjectionMakerTest.java @@ -0,0 +1,745 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ + +package com.yahoo.elide.jsonapi; + +import static org.junit.jupiter.api.Assertions.assertEquals; + +import com.yahoo.elide.core.EntityDictionary; +import com.yahoo.elide.core.Path; +import com.yahoo.elide.core.RequestScope; +import com.yahoo.elide.core.TestRequestScope; +import com.yahoo.elide.core.filter.InInsensitivePredicate; +import com.yahoo.elide.core.filter.expression.FilterExpression; +import com.yahoo.elide.core.pagination.Pagination; +import com.yahoo.elide.core.sort.Sorting; +import com.yahoo.elide.request.Attribute; +import com.yahoo.elide.request.EntityProjection; +import example.Address; +import example.Author; +import example.Book; +import example.Editor; +import example.Publisher; +import org.junit.jupiter.api.BeforeAll; +import org.junit.jupiter.api.Test; +import org.junit.jupiter.api.TestInstance; + +import java.util.HashMap; + +import javax.ws.rs.core.MultivaluedHashMap; +import javax.ws.rs.core.MultivaluedMap; + + +@TestInstance(TestInstance.Lifecycle.PER_CLASS) +public class EntityProjectionMakerTest { + private EntityDictionary dictionary; + + @BeforeAll + public void init() { + dictionary = new EntityDictionary(new HashMap<>()); + dictionary.bindEntity(Book.class); + dictionary.bindEntity(Author.class); + dictionary.bindEntity(Publisher.class); + dictionary.bindEntity(Editor.class); + } + + @Test + public void testRootCollectionNoQueryParams() { + MultivaluedMap queryParams = new MultivaluedHashMap<>(); + String path = "/book"; + + RequestScope scope = new TestRequestScope(dictionary, path, queryParams); + Pagination defaultPagination = scope.getPagination(); + + EntityProjectionMaker maker = new EntityProjectionMaker(dictionary, scope); + + EntityProjection expected = EntityProjection.builder() + .type(Book.class) + .attribute(Attribute.builder().name("title").type(String.class).build()) + .attribute(Attribute.builder().name("genre").type(String.class).build()) + .attribute(Attribute.builder().name("language").type(String.class).build()) + .attribute(Attribute.builder().name("publishDate").type(long.class).build()) + .relationship("authors", EntityProjection.builder() + .type(Author.class) + .build()) + .relationship("publisher", EntityProjection.builder() + .type(Publisher.class) + .build()) + .relationship("editor", EntityProjection.builder() + .type(Editor.class) + .build()) + .pagination(defaultPagination) + .build(); + + EntityProjection actual = maker.parsePath(path); + + assertEquals(expected, actual); + } + + @Test + public void testRootCollectionSparseFields() { + MultivaluedMap queryParams = new MultivaluedHashMap<>(); + queryParams.add("fields[book]", "title,publishDate,authors"); + String path = "/book"; + + RequestScope scope = new TestRequestScope(dictionary, path, queryParams); + Pagination defaultPagination = scope.getPagination(); + + EntityProjectionMaker maker = new EntityProjectionMaker(dictionary, scope); + + EntityProjection expected = EntityProjection.builder() + .type(Book.class) + .attribute(Attribute.builder().name("title").type(String.class).build()) + .attribute(Attribute.builder().name("publishDate").type(long.class).build()) + .relationship("authors", EntityProjection.builder() + .type(Author.class) + .build()) + .pagination(defaultPagination) + .build(); + + EntityProjection actual = maker.parsePath(path); + + assertEquals(expected, actual); + } + + @Test + public void testRootEntityNoQueryParams() { + MultivaluedMap queryParams = new MultivaluedHashMap<>(); + String path = "/book/1"; + + RequestScope scope = new TestRequestScope(dictionary, path, queryParams); + Pagination defaultPagination = scope.getPagination(); + + EntityProjectionMaker maker = new EntityProjectionMaker(dictionary, scope); + + EntityProjection expected = EntityProjection.builder() + .type(Book.class) + .attribute(Attribute.builder().name("title").type(String.class).build()) + .attribute(Attribute.builder().name("genre").type(String.class).build()) + .attribute(Attribute.builder().name("language").type(String.class).build()) + .attribute(Attribute.builder().name("publishDate").type(long.class).build()) + .relationship("authors", EntityProjection.builder() + .type(Author.class) + .build()) + .relationship("publisher", EntityProjection.builder() + .type(Publisher.class) + .build()) + .relationship("editor", EntityProjection.builder() + .type(Editor.class) + .build()) + .build(); + + EntityProjection actual = maker.parsePath(path); + + assertEquals(expected, actual); + } + + @Test + public void testNestedCollectionNoQueryParams() { + MultivaluedMap queryParams = new MultivaluedHashMap<>(); + String path = "/author/1/books/3/publisher"; + + RequestScope scope = new TestRequestScope(dictionary, path, queryParams); + Pagination defaultPagination = scope.getPagination(); + + EntityProjectionMaker maker = new EntityProjectionMaker(dictionary, scope); + + EntityProjection expected = EntityProjection.builder() + .type(Author.class) + .relationship("books", EntityProjection.builder() + .type(Book.class) + .relationship("publisher", EntityProjection.builder() + .type(Publisher.class) + .attribute(Attribute.builder().name("name").type(String.class).build()) + .attribute(Attribute.builder().name("updateHookInvoked").type(boolean.class).build()) + .relationship("books", EntityProjection.builder() + .type(Book.class) + .build()) + .relationship("editor", EntityProjection.builder() + .type(Editor.class) + .build()) + .pagination(defaultPagination) + .build()) + .build()) + .build(); + + EntityProjection actual = maker.parsePath(path); + + assertEquals(expected, actual); + } + + @Test + public void testNestedEntityNoQueryParams() { + MultivaluedMap queryParams = new MultivaluedHashMap<>(); + String path = "/author/1/books/3/publisher/1"; + + RequestScope scope = new TestRequestScope(dictionary, path, queryParams); + Pagination defaultPagination = scope.getPagination(); + + EntityProjectionMaker maker = new EntityProjectionMaker(dictionary, scope); + + EntityProjection expected = EntityProjection.builder() + .type(Author.class) + .relationship("books", EntityProjection.builder() + .type(Book.class) + .relationship("publisher", EntityProjection.builder() + .type(Publisher.class) + .attribute(Attribute.builder().name("name").type(String.class).build()) + .attribute(Attribute.builder().name("updateHookInvoked").type(boolean.class).build()) + .relationship("books", EntityProjection.builder() + .type(Book.class) + .build()) + .relationship("editor", EntityProjection.builder() + .type(Editor.class) + .build()) + .build()) + .build()) + .build(); + + EntityProjection actual = maker.parsePath(path); + + assertEquals(expected, actual); + } + + @Test + public void testRelationshipNoQueryParams() { + MultivaluedMap queryParams = new MultivaluedHashMap<>(); + String path = "/author/1/relationships/books"; + + RequestScope scope = new TestRequestScope(dictionary, path, queryParams); + Pagination defaultPagination = scope.getPagination(); + + EntityProjectionMaker maker = new EntityProjectionMaker(dictionary, scope); + + EntityProjection expected = EntityProjection.builder() + .type(Author.class) + .relationship("books", EntityProjection.builder() + .type(Book.class) + .pagination(defaultPagination) + .build()) + .build(); + + EntityProjection actual = maker.parsePath(path); + + assertEquals(expected, actual); + } + + @Test + public void testRelationshipWithSingleInclude() { + MultivaluedMap queryParams = new MultivaluedHashMap<>(); + queryParams.add("include", "authors"); + String path = "/book/1/relationships/publisher"; + + RequestScope scope = new TestRequestScope(dictionary, path, queryParams); + Pagination defaultPagination = scope.getPagination(); + + EntityProjectionMaker maker = new EntityProjectionMaker(dictionary, scope); + + EntityProjection expected = EntityProjection.builder() + .type(Book.class) + .relationship("publisher", EntityProjection.builder() + .type(Publisher.class) + .pagination(defaultPagination) + .build()) + .relationship("authors", EntityProjection.builder() + .attribute(Attribute.builder().name("name").type(String.class).build()) + .attribute(Attribute.builder().name("type").type(Author.AuthorType.class).build()) + .attribute(Attribute.builder().name("homeAddress").type(Address.class).build()) + .relationship("books", EntityProjection.builder() + .type(Book.class) + .build()) + .type(Author.class) + .build()) + .relationship("editor", EntityProjection.builder() + .type(Editor.class) + .build()) + .build(); + + EntityProjection actual = maker.parsePath(path); + + assertEquals(expected, actual); + } + + @Test + public void testRootCollectionWithSingleInclude() { + MultivaluedMap queryParams = new MultivaluedHashMap<>(); + queryParams.add("include", "authors"); + String path = "/book"; + + RequestScope scope = new TestRequestScope(dictionary, path, queryParams); + Pagination defaultPagination = scope.getPagination(); + + EntityProjectionMaker maker = new EntityProjectionMaker(dictionary, scope); + + EntityProjection expected = EntityProjection.builder() + .type(Book.class) + .attribute(Attribute.builder().name("title").type(String.class).build()) + .attribute(Attribute.builder().name("genre").type(String.class).build()) + .attribute(Attribute.builder().name("language").type(String.class).build()) + .attribute(Attribute.builder().name("publishDate").type(long.class).build()) + .relationship("authors", EntityProjection.builder() + .type(Author.class) + .attribute(Attribute.builder().name("name").type(String.class).build()) + .attribute(Attribute.builder().name("type").type(Author.AuthorType.class).build()) + .attribute(Attribute.builder().name("homeAddress").type(Address.class).build()) + .relationship("books", EntityProjection.builder() + .type(Book.class) + .build()) + .build()) + .relationship("publisher", EntityProjection.builder() + .type(Publisher.class) + .build()) + .relationship("editor", EntityProjection.builder() + .type(Editor.class) + .build()) + .pagination(defaultPagination) + .build(); + + EntityProjection actual = maker.parsePath(path); + + assertEquals(expected, actual); + } + + @Test + public void testRootEntityWithSingleInclude() { + MultivaluedMap queryParams = new MultivaluedHashMap<>(); + queryParams.add("include", "authors"); + String path = "/book/1"; + + RequestScope scope = new TestRequestScope(dictionary, path, queryParams); + Pagination defaultPagination = scope.getPagination(); + + EntityProjectionMaker maker = new EntityProjectionMaker(dictionary, scope); + + EntityProjection expected = EntityProjection.builder() + .type(Book.class) + .attribute(Attribute.builder().name("title").type(String.class).build()) + .attribute(Attribute.builder().name("genre").type(String.class).build()) + .attribute(Attribute.builder().name("language").type(String.class).build()) + .attribute(Attribute.builder().name("publishDate").type(long.class).build()) + .relationship("authors", EntityProjection.builder() + .type(Author.class) + .attribute(Attribute.builder().name("name").type(String.class).build()) + .attribute(Attribute.builder().name("type").type(Author.AuthorType.class).build()) + .attribute(Attribute.builder().name("homeAddress").type(Address.class).build()) + .relationship("books", EntityProjection.builder() + .type(Book.class) + .build()) + .build()) + .relationship("publisher", EntityProjection.builder() + .type(Publisher.class) + .build()) + .relationship("editor", EntityProjection.builder() + .type(Editor.class) + .build()) + .build(); + + EntityProjection actual = maker.parsePath(path); + + assertEquals(expected, actual); + } + + @Test + public void testRootCollectionWithNestedInclude() throws Exception { + MultivaluedMap queryParams = new MultivaluedHashMap<>(); + queryParams.add("include", "books"); + queryParams.add("include", "books.publisher,books.editor"); + String path = "/author"; + + RequestScope scope = new TestRequestScope(dictionary, path, queryParams); + Pagination defaultPagination = scope.getPagination(); + + EntityProjectionMaker maker = new EntityProjectionMaker(dictionary, scope); + + EntityProjection expected = EntityProjection.builder() + .type(Author.class) + .attribute(Attribute.builder().name("homeAddress").type(Address.class).build()) + .attribute(Attribute.builder().name("name").type(String.class).build()) + .attribute(Attribute.builder().name("type").type(Author.AuthorType.class).build()) + .relationship("books", EntityProjection.builder() + .type(Book.class) + .attribute(Attribute.builder().name("title").type(String.class).build()) + .attribute(Attribute.builder().name("genre").type(String.class).build()) + .attribute(Attribute.builder().name("language").type(String.class).build()) + .attribute(Attribute.builder().name("publishDate").type(long.class).build()) + .relationship("editor", EntityProjection.builder() + .type(Editor.class) + .attribute(Attribute.builder().name("firstName").type(String.class).build()) + .attribute(Attribute.builder().name("lastName").type(String.class).build()) + .attribute(Attribute.builder().name("fullName").type(String.class).build()) + .relationship("editor", EntityProjection.builder() + .type(Editor.class) + .build()) + .build()) + .relationship("publisher", EntityProjection.builder() + .type(Publisher.class) + .attribute(Attribute.builder().name("name").type(String.class).build()) + .attribute(Attribute.builder().name("updateHookInvoked").type(boolean.class).build()) + .relationship("books", EntityProjection.builder() + .type(Book.class) + .build()) + .relationship("editor", EntityProjection.builder() + .type(Editor.class) + .build()) + .build()) + .relationship("authors", EntityProjection.builder() + .type(Author.class) + .build()) + .build()) + .pagination(defaultPagination) + .build(); + + EntityProjection actual = maker.parsePath(path); + + assertEquals(expected, actual); + } + + @Test + public void testRootEntityWithNestedInclude() { + MultivaluedMap queryParams = new MultivaluedHashMap<>(); + queryParams.add("include", "books"); + queryParams.add("include", "books.publisher,books.editor"); + String path = "/author/1"; + + RequestScope scope = new TestRequestScope(dictionary, path, queryParams); + Pagination defaultPagination = scope.getPagination(); + + EntityProjectionMaker maker = new EntityProjectionMaker(dictionary, scope); + + EntityProjection expected = EntityProjection.builder() + .type(Author.class) + .attribute(Attribute.builder().name("name").type(String.class).build()) + .attribute(Attribute.builder().name("type").type(Author.AuthorType.class).build()) + .attribute(Attribute.builder().name("homeAddress").type(Address.class).build()) + .relationship("books", EntityProjection.builder() + .type(Book.class) + .attribute(Attribute.builder().name("title").type(String.class).build()) + .attribute(Attribute.builder().name("genre").type(String.class).build()) + .attribute(Attribute.builder().name("language").type(String.class).build()) + .attribute(Attribute.builder().name("publishDate").type(long.class).build()) + .relationship("publisher", EntityProjection.builder() + .type(Publisher.class) + .attribute(Attribute.builder().name("name").type(String.class).build()) + .attribute(Attribute.builder().name("updateHookInvoked").type(boolean.class).build()) + .relationship("books", EntityProjection.builder() + .type(Book.class) + .build()) + .relationship("editor", EntityProjection.builder() + .type(Editor.class) + .build()) + .build()) + .relationship("editor", EntityProjection.builder() + .type(Editor.class) + .attribute(Attribute.builder().name("firstName").type(String.class).build()) + .attribute(Attribute.builder().name("lastName").type(String.class).build()) + .attribute(Attribute.builder().name("fullName").type(String.class).build()) + .relationship("editor", EntityProjection.builder() + .type(Editor.class) + .build()) + .build()) + .relationship("authors", EntityProjection.builder() + .type(Author.class) + .build()) + .build()) + .build(); + + EntityProjection actual = maker.parsePath(path); + + assertEquals(expected, actual); + } + + @Test + public void testNestedEntityWithSingleInclude() { + MultivaluedMap queryParams = new MultivaluedHashMap<>(); + queryParams.add("include", "books"); + String path = "/author/1/books/3/publisher/1"; + + RequestScope scope = new TestRequestScope(dictionary, path, queryParams); + Pagination defaultPagination = scope.getPagination(); + + EntityProjectionMaker maker = new EntityProjectionMaker(dictionary, scope); + + EntityProjection expected = EntityProjection.builder() + .type(Author.class) + .relationship("books", EntityProjection.builder() + .type(Book.class) + .relationship("publisher", EntityProjection.builder() + .type(Publisher.class) + .attribute(Attribute.builder().name("name").type(String.class).build()) + .attribute(Attribute.builder().name("updateHookInvoked").type(boolean.class).build()) + .relationship("books", EntityProjection.builder() + .type(Book.class) + .attribute(Attribute.builder().name("title").type(String.class).build()) + .attribute(Attribute.builder().name("genre").type(String.class).build()) + .attribute(Attribute.builder().name("language").type(String.class).build()) + .attribute(Attribute.builder().name("publishDate").type(long.class).build()) + .relationship("authors", EntityProjection.builder() + .type(Author.class) + .build()) + .relationship("publisher", EntityProjection.builder() + .type(Publisher.class) + .build()) + .relationship("editor", EntityProjection.builder() + .type(Editor.class) + .build()) + .build()) + .relationship("editor", EntityProjection.builder() + .type(Editor.class) + .build()) + .build()) + .build()) + .build(); + + EntityProjection actual = maker.parsePath(path); + + assertEquals(expected, actual); + } + + @Test + public void testNestedCollectionWithSingleInclude() { + MultivaluedMap queryParams = new MultivaluedHashMap<>(); + queryParams.add("include", "books"); + String path = "/author/1/books/3/publisher"; + + RequestScope scope = new TestRequestScope(dictionary, path, queryParams); + Pagination defaultPagination = scope.getPagination(); + + EntityProjectionMaker maker = new EntityProjectionMaker(dictionary, scope); + + EntityProjection expected = EntityProjection.builder() + .type(Author.class) + .relationship("books", EntityProjection.builder() + .type(Book.class) + .relationship("publisher", EntityProjection.builder() + .type(Publisher.class) + .attribute(Attribute.builder().name("name").type(String.class).build()) + .attribute(Attribute.builder().name("updateHookInvoked").type(boolean.class).build()) + .relationship("books", EntityProjection.builder() + .attribute(Attribute.builder().name("title").type(String.class).build()) + .attribute(Attribute.builder().name("genre").type(String.class).build()) + .attribute(Attribute.builder().name("language").type(String.class).build()) + .attribute(Attribute.builder().name("publishDate").type(long.class).build()) + .relationship("authors", EntityProjection.builder() + .type(Author.class) + .build()) + .relationship("publisher", EntityProjection.builder() + .type(Publisher.class) + .build()) + .relationship("editor", EntityProjection.builder() + .type(Editor.class) + .build()) + .type(Book.class) + .build()) + .relationship("editor", EntityProjection.builder() + .type(Editor.class) + .build()) + .pagination(defaultPagination) + .build()) + .build()) + .build(); + + EntityProjection actual = maker.parsePath(path); + + assertEquals(expected, actual); + } + + @Test + public void testRootEntityWithNestedIncludeAndSparseFields() { + MultivaluedMap queryParams = new MultivaluedHashMap<>(); + queryParams.add("include", "books"); + queryParams.add("include", "books.publisher,books.editor"); + queryParams.add("fields[publisher]", "name"); + queryParams.add("fields[editor]", "fullName"); + queryParams.add("fields[book]", "publisher,editor,title"); + String path = "/author/1"; + + RequestScope scope = new TestRequestScope(dictionary, path, queryParams); + Pagination defaultPagination = scope.getPagination(); + + EntityProjectionMaker maker = new EntityProjectionMaker(dictionary, scope); + + EntityProjection expected = EntityProjection.builder() + .type(Author.class) + .attribute(Attribute.builder().name("name").type(String.class).build()) + .attribute(Attribute.builder().name("type").type(Author.AuthorType.class).build()) + .attribute(Attribute.builder().name("homeAddress").type(Address.class).build()) + .relationship("books", EntityProjection.builder() + .type(Book.class) + .attribute(Attribute.builder().name("title").type(String.class).build()) + .relationship("publisher", EntityProjection.builder() + .type(Publisher.class) + .attribute(Attribute.builder().name("name").type(String.class).build()) + .build()) + .relationship("editor", EntityProjection.builder() + .type(Editor.class) + .attribute(Attribute.builder().name("fullName").type(String.class).build()) + .build()) + .build()) + .build(); + + EntityProjection actual = maker.parsePath(path); + + assertEquals(expected, actual); + } + + @Test + public void testRootCollectionWithGlobalFilter() { + MultivaluedMap queryParams = new MultivaluedHashMap<>(); + queryParams.add("filter", "genre=='Science Fiction'"); + String path = "/book"; + + RequestScope scope = new TestRequestScope(dictionary, path, queryParams); + Pagination defaultPagination = scope.getPagination(); + + FilterExpression expression = + new InInsensitivePredicate(new Path(Book.class, dictionary, "genre"), "Science Fiction"); + + EntityProjectionMaker maker = new EntityProjectionMaker(dictionary, scope); + + EntityProjection expected = EntityProjection.builder() + .type(Book.class) + .attribute(Attribute.builder().name("title").type(String.class).build()) + .attribute(Attribute.builder().name("genre").type(String.class).build()) + .attribute(Attribute.builder().name("language").type(String.class).build()) + .attribute(Attribute.builder().name("publishDate").type(long.class).build()) + .filterExpression(expression) + .relationship("authors", EntityProjection.builder() + .type(Author.class) + .build()) + .relationship("publisher", EntityProjection.builder() + .type(Publisher.class) + .build()) + .relationship("editor", EntityProjection.builder() + .type(Editor.class) + .build()) + .pagination(defaultPagination) + .build(); + + EntityProjection actual = maker.parsePath(path); + + assertEquals(expected, actual); + } + + @Test + public void testNestedCollectionWithTypedFilter() { + MultivaluedMap queryParams = new MultivaluedHashMap<>(); + queryParams.add("filter[publisher]", "name=='Foo'"); + String path = "/author/1/books/3/publisher"; + + FilterExpression expression = + new InInsensitivePredicate(new Path(Publisher.class, dictionary, "name"), "Foo"); + + RequestScope scope = new TestRequestScope(dictionary, path, queryParams); + Pagination defaultPagination = scope.getPagination(); + + EntityProjectionMaker maker = new EntityProjectionMaker(dictionary, scope); + + EntityProjection expected = EntityProjection.builder() + .type(Author.class) + .relationship("books", EntityProjection.builder() + .type(Book.class) + .relationship("publisher", EntityProjection.builder() + .type(Publisher.class) + .attribute(Attribute.builder().name("name").type(String.class).build()) + .attribute(Attribute.builder().name("updateHookInvoked").type(boolean.class).build()) + .filterExpression(expression) + .relationship("books", EntityProjection.builder() + .type(Book.class) + .build()) + .relationship("editor", EntityProjection.builder() + .type(Editor.class) + .build()) + .pagination(defaultPagination) + .build()) + .build()) + .build(); + + EntityProjection actual = maker.parsePath(path); + + assertEquals(expected, actual); + } + + @Test + public void testRelationshipsAndIncludeWithFilterAndSort() { + MultivaluedMap queryParams = new MultivaluedHashMap<>(); + queryParams.add("include", "authors"); + queryParams.add("filter[author]", "name=='Foo'"); + queryParams.add("filter[publisher]", "name=='Foo'"); + queryParams.add("sort", "name"); + String path = "/book/1/relationships/publisher"; + Sorting sorting = Sorting.parseSortRule("name"); + + RequestScope scope = new TestRequestScope(dictionary, path, queryParams); + Pagination defaultPagination = scope.getPagination(); + + EntityProjectionMaker maker = new EntityProjectionMaker(dictionary, scope); + + EntityProjection expected = EntityProjection.builder() + .type(Book.class) + .relationship("publisher", EntityProjection.builder() + .type(Publisher.class) + .filterExpression(new InInsensitivePredicate(new Path(Publisher.class, dictionary, "name"), "Foo")) + .sorting(sorting) + .pagination(defaultPagination) + .build()) + .relationship("authors", EntityProjection.builder() + .attribute(Attribute.builder().name("name").type(String.class).build()) + .attribute(Attribute.builder().name("type").type(Author.AuthorType.class).build()) + .attribute(Attribute.builder().name("homeAddress").type(Address.class).build()) + .filterExpression(new InInsensitivePredicate(new Path(Author.class, dictionary, "name"), "Foo")) + .relationship("books", EntityProjection.builder() + .type(Book.class) + .build()) + .type(Author.class) + .build()) + .relationship("editor", EntityProjection.builder() + .type(Editor.class) + .build()) + .build(); + + EntityProjection actual = maker.parsePath(path); + + assertEquals(expected, actual); + } + + @Test + public void testRootCollectionWithTypedFilter() { + MultivaluedMap queryParams = new MultivaluedHashMap<>(); + queryParams.add("filter[book]", "genre=='Science Fiction'"); + String path = "/book"; + + RequestScope scope = new TestRequestScope(dictionary, path, queryParams); + Pagination defaultPagination = scope.getPagination(); + + FilterExpression expression = + new InInsensitivePredicate(new Path(Book.class, dictionary, "genre"), "Science Fiction"); + + EntityProjectionMaker maker = new EntityProjectionMaker(dictionary, scope); + + EntityProjection expected = EntityProjection.builder() + .type(Book.class) + .attribute(Attribute.builder().name("title").type(String.class).build()) + .attribute(Attribute.builder().name("genre").type(String.class).build()) + .attribute(Attribute.builder().name("language").type(String.class).build()) + .attribute(Attribute.builder().name("publishDate").type(long.class).build()) + .filterExpression(expression) + .relationship("authors", EntityProjection.builder() + .type(Author.class) + .build()) + .relationship("publisher", EntityProjection.builder() + .type(Publisher.class) + .build()) + .relationship("editor", EntityProjection.builder() + .type(Editor.class) + .build()) + .pagination(defaultPagination) + .build(); + + EntityProjection actual = maker.parsePath(path); + + assertEquals(expected, actual); + } +} diff --git a/elide-core/src/test/java/com/yahoo/elide/jsonapi/JsonApiTest.java b/elide-core/src/test/java/com/yahoo/elide/jsonapi/JsonApiTest.java index 31aa9c7093..0a2b134d00 100644 --- a/elide-core/src/test/java/com/yahoo/elide/jsonapi/JsonApiTest.java +++ b/elide-core/src/test/java/com/yahoo/elide/jsonapi/JsonApiTest.java @@ -7,16 +7,14 @@ import static org.junit.jupiter.api.Assertions.assertEquals; import static org.junit.jupiter.api.Assertions.assertNull; -import static org.junit.jupiter.api.Assertions.assertTrue; import static org.mockito.Mockito.mock; -import com.yahoo.elide.ElideSettingsBuilder; -import com.yahoo.elide.audit.AuditLogger; -import com.yahoo.elide.audit.TestAuditLogger; import com.yahoo.elide.core.DataStoreTransaction; import com.yahoo.elide.core.EntityDictionary; import com.yahoo.elide.core.PersistentResource; import com.yahoo.elide.core.RequestScope; +import com.yahoo.elide.core.TestDictionary; +import com.yahoo.elide.core.TestRequestScope; import com.yahoo.elide.jsonapi.models.Data; import com.yahoo.elide.jsonapi.models.JsonApiDocument; import com.yahoo.elide.jsonapi.models.Meta; @@ -31,10 +29,8 @@ import example.Child; import example.Parent; -import example.TestCheckMappings; - import org.apache.commons.collections4.IterableUtils; -import org.junit.jupiter.api.BeforeAll; +import org.junit.jupiter.api.BeforeEach; import org.junit.jupiter.api.Test; import org.mockito.Answers; @@ -50,37 +46,17 @@ * JSON API testing. */ public class JsonApiTest { - private static RequestScope userScope; - private static JsonApiMapper mapper; - - @BeforeAll - static void init() { - EntityDictionary dictionary = new EntityDictionary(TestCheckMappings.MAPPINGS); + private JsonApiMapper mapper; + private User user = new User(0); + private EntityDictionary dictionary; + private DataStoreTransaction tx = mock(DataStoreTransaction.class, Answers.CALLS_REAL_METHODS); + + @BeforeEach + void init() { + dictionary = TestDictionary.getTestDictionary(); dictionary.bindEntity(Parent.class); dictionary.bindEntity(Child.class); - dictionary.bindInitializer(Parent::doInit, Parent.class); mapper = new JsonApiMapper(dictionary); - AuditLogger testLogger = new TestAuditLogger(); - userScope = new RequestScope(null, new JsonApiDocument(), - mock(DataStoreTransaction.class, Answers.CALLS_REAL_METHODS), new User(0), null, - new ElideSettingsBuilder(null) - .withJsonApiMapper(mapper) - .withAuditLogger(testLogger) - .withEntityDictionary(dictionary) - .build()); - } - - @Test - public void checkInit() { - // Ensure that our object receives its init before serializing - Parent parent = new Parent(); - parent.setId(123L); - parent.setChildren(Sets.newHashSet()); - parent.setSpouses(Sets.newHashSet()); - - new PersistentResource<>(parent, null, userScope.getUUIDFor(parent), userScope).toResource(); - - assertTrue(parent.init); } @Test @@ -88,6 +64,8 @@ public void writeSingleNoAttributesNoRel() throws JsonProcessingException { Parent parent = new Parent(); parent.setId(123L); + RequestScope userScope = new TestRequestScope(tx, user, dictionary); + JsonApiDocument jsonApiDocument = new JsonApiDocument(); jsonApiDocument.setData(new Data<>(new PersistentResource<>(parent, null, userScope.getUUIDFor(parent), userScope).toResource())); @@ -112,6 +90,8 @@ public void writeSingle() throws JsonProcessingException { child.setParents(Collections.singleton(parent)); child.setFriends(new HashSet<>()); + RequestScope userScope = new TestRequestScope(tx, user, dictionary); + JsonApiDocument jsonApiDocument = new JsonApiDocument(); jsonApiDocument.setData(new Data<>(new PersistentResource<>(parent, null, userScope.getUUIDFor(parent), userScope).toResource())); @@ -136,11 +116,14 @@ public void writeSingleIncluded() throws JsonProcessingException { child.setParents(Collections.singleton(parent)); child.setFriends(new HashSet<>()); + RequestScope userScope = new TestRequestScope(tx, user, dictionary); + PersistentResource pRec = new PersistentResource<>(parent, null, userScope.getUUIDFor(parent), userScope); JsonApiDocument jsonApiDocument = new JsonApiDocument(); jsonApiDocument.setData(new Data<>(pRec.toResource())); - jsonApiDocument.addIncluded(new PersistentResource<>(child, pRec, userScope.getUUIDFor(child), userScope).toResource()); + jsonApiDocument.addIncluded( + new PersistentResource<>(child, pRec, userScope.getUUIDFor(child), userScope).toResource()); String expected = "{\"data\":{\"type\":\"parent\",\"id\":\"123\",\"attributes\":{\"firstName\":\"bob\"},\"relationships\":{\"children\":{\"data\":[{\"type\":\"child\",\"id\":\"2\"}]},\"spouses\":{\"data\":[]}}},\"included\":[{\"type\":\"child\",\"id\":\"2\",\"attributes\":{\"name\":null},\"relationships\":{\"friends\":{\"data\":[]},\"parents\":{\"data\":[{\"type\":\"parent\",\"id\":\"123\"}]}}}]}"; @@ -164,6 +147,8 @@ public void writeList() throws JsonProcessingException { parent.setFirstName("bob"); child.setFriends(new HashSet<>()); + RequestScope userScope = new TestRequestScope(tx, user, dictionary); + JsonApiDocument jsonApiDocument = new JsonApiDocument(); jsonApiDocument.setData( new Data<>(Collections.singletonList(new PersistentResource<>(parent, null, userScope.getUUIDFor(parent), userScope).toResource()))); @@ -189,13 +174,17 @@ public void writeListIncluded() throws JsonProcessingException { parent.setFirstName("bob"); child.setFriends(new HashSet<>()); + RequestScope userScope = new TestRequestScope(tx, user, dictionary); + PersistentResource pRec = new PersistentResource<>(parent, null, userScope.getUUIDFor(parent), userScope); JsonApiDocument jsonApiDocument = new JsonApiDocument(); jsonApiDocument.setData(new Data<>(Collections.singletonList(pRec.toResource()))); - jsonApiDocument.addIncluded(new PersistentResource<>(child, pRec, userScope.getUUIDFor(child), userScope).toResource()); + jsonApiDocument.addIncluded(new PersistentResource<>(child, + pRec, userScope.getUUIDFor(child), userScope).toResource()); // duplicate will be ignored - jsonApiDocument.addIncluded(new PersistentResource<>(child, pRec, userScope.getUUIDFor(child), userScope).toResource()); + jsonApiDocument.addIncluded( + new PersistentResource<>(child, pRec, userScope.getUUIDFor(child), userScope).toResource()); String expected = "{\"data\":[{\"type\":\"parent\",\"id\":\"123\",\"attributes\":{\"firstName\":\"bob\"},\"relationships\":{\"children\":{\"data\":[{\"type\":\"child\",\"id\":\"2\"}]},\"spouses\":{\"data\":[]}}}],\"included\":[{\"type\":\"child\",\"id\":\"2\",\"attributes\":{\"name\":null},\"relationships\":{\"friends\":{\"data\":[]},\"parents\":{\"data\":[{\"type\":\"parent\",\"id\":\"123\"}]}}}]}"; diff --git a/elide-core/src/test/java/com/yahoo/elide/jsonapi/document/processors/IncludedProcessorTest.java b/elide-core/src/test/java/com/yahoo/elide/jsonapi/document/processors/IncludedProcessorTest.java index c835d130a5..bf3e052990 100644 --- a/elide-core/src/test/java/com/yahoo/elide/jsonapi/document/processors/IncludedProcessorTest.java +++ b/elide-core/src/test/java/com/yahoo/elide/jsonapi/document/processors/IncludedProcessorTest.java @@ -8,14 +8,13 @@ import static org.junit.jupiter.api.Assertions.assertEquals; import static org.junit.jupiter.api.Assertions.assertNull; import static org.mockito.Mockito.mock; +import static org.mockito.Mockito.reset; -import com.yahoo.elide.ElideSettings; -import com.yahoo.elide.ElideSettingsBuilder; -import com.yahoo.elide.audit.TestAuditLogger; import com.yahoo.elide.core.DataStoreTransaction; import com.yahoo.elide.core.EntityDictionary; import com.yahoo.elide.core.PersistentResource; -import com.yahoo.elide.core.RequestScope; +import com.yahoo.elide.core.TestDictionary; +import com.yahoo.elide.core.TestRequestScope; import com.yahoo.elide.jsonapi.models.JsonApiDocument; import com.yahoo.elide.jsonapi.models.Resource; import com.yahoo.elide.security.User; @@ -24,7 +23,6 @@ import example.Child; import example.FunWithPermissions; import example.Parent; -import example.TestCheckMappings; import org.junit.jupiter.api.BeforeEach; import org.junit.jupiter.api.Test; import org.mockito.Answers; @@ -38,7 +36,6 @@ import javax.ws.rs.core.MultivaluedHashMap; import javax.ws.rs.core.MultivaluedMap; - public class IncludedProcessorTest { private static final String INCLUDE = "include"; @@ -55,29 +52,22 @@ public class IncludedProcessorTest { private PersistentResource funWithPermissionsRecord; + private DataStoreTransaction mockTransaction = mock(DataStoreTransaction.class, Answers.CALLS_REAL_METHODS); + private TestRequestScope testScope; + private EntityDictionary dictionary; + @BeforeEach public void setUp() throws Exception { includedProcessor = new IncludedProcessor(); - EntityDictionary dictionary = new EntityDictionary(TestCheckMappings.MAPPINGS); + dictionary = TestDictionary.getTestDictionary(); dictionary.bindEntity(Child.class); dictionary.bindEntity(Parent.class); dictionary.bindEntity(FunWithPermissions.class); - ElideSettings elideSettings = new ElideSettingsBuilder(null) - .withAuditLogger(new TestAuditLogger()) - .withEntityDictionary(dictionary) - .build(); - - RequestScope goodUserScope = new RequestScope(null, - new JsonApiDocument(), mock(DataStoreTransaction.class, Answers.CALLS_REAL_METHODS), - new User(1), null, - elideSettings); + reset(mockTransaction); - RequestScope badUserScope = new RequestScope(null, - new JsonApiDocument(), mock(DataStoreTransaction.class, Answers.CALLS_REAL_METHODS), - new User(-1), null, - elideSettings); + testScope = new TestRequestScope(mockTransaction, new User(1), dictionary); //Create objects Parent parent1 = newParent(1); @@ -102,16 +92,17 @@ public void setUp() throws Exception { child3.setFriends(new HashSet<>(Collections.singletonList(child1))); child4.setFriends(new HashSet<>(Collections.singletonList(child2))); - //Create Persistent Resources - parentRecord1 = new PersistentResource<>(parent1, null, goodUserScope.getUUIDFor(parent1), goodUserScope); - parentRecord2 = new PersistentResource<>(parent2, null, goodUserScope.getUUIDFor(parent2), goodUserScope); - parentRecord3 = new PersistentResource<>(parent3, null, goodUserScope.getUUIDFor(parent3), goodUserScope); - childRecord1 = new PersistentResource<>(child1, null, goodUserScope.getUUIDFor(child1), goodUserScope); - childRecord2 = new PersistentResource<>(child2, null, goodUserScope.getUUIDFor(child2), goodUserScope); - childRecord3 = new PersistentResource<>(child3, null, goodUserScope.getUUIDFor(child3), goodUserScope); - childRecord4 = new PersistentResource<>(child4, null, goodUserScope.getUUIDFor(child4), goodUserScope); - - funWithPermissionsRecord = new PersistentResource<>(funWithPermissions, null, goodUserScope.getUUIDFor(funWithPermissions), badUserScope); + //Create Persistent Resource + parentRecord1 = new PersistentResource<>(parent1, null, String.valueOf(parent1.getId()), testScope); + parentRecord2 = new PersistentResource<>(parent2, null, String.valueOf(parent2.getId()), testScope); + parentRecord3 = new PersistentResource<>(parent3, null, String.valueOf(parent3.getId()), testScope); + childRecord1 = new PersistentResource<>(child1, null, String.valueOf(child1.getId()), testScope); + childRecord2 = new PersistentResource<>(child2, null, String.valueOf(child2.getId()), testScope); + childRecord3 = new PersistentResource<>(child3, null, String.valueOf(child3.getId()), testScope); + childRecord4 = new PersistentResource<>(child4, null, String.valueOf(child4.getId()), testScope); + + funWithPermissionsRecord = new PersistentResource<>(funWithPermissions, null, + String.valueOf(funWithPermissions.getId()), testScope); } @Test @@ -120,6 +111,7 @@ public void testExecuteSingleRelation() throws Exception { MultivaluedMap queryParams = new MultivaluedHashMap<>(); queryParams.put(INCLUDE, Collections.singletonList("children")); + testScope.setQueryParams(queryParams); includedProcessor.execute(jsonApiDocument, parentRecord1, Optional.of(queryParams)); List expectedIncluded = Collections.singletonList(childRecord1.toResource()); @@ -139,6 +131,7 @@ public void testExecuteSingleRelationOnCollection() throws Exception { MultivaluedMap queryParams = new MultivaluedHashMap<>(); queryParams.put(INCLUDE, Collections.singletonList("children")); + testScope.setQueryParams(queryParams); includedProcessor.execute(jsonApiDocument, parents, Optional.of(queryParams)); List expectedIncluded = Arrays.asList(childRecord1.toResource(), childRecord2.toResource()); @@ -150,10 +143,12 @@ public void testExecuteSingleRelationOnCollection() throws Exception { @Test public void testExecuteSingleNestedRelation() throws Exception { + JsonApiDocument jsonApiDocument = new JsonApiDocument(); MultivaluedMap queryParams = new MultivaluedHashMap<>(); queryParams.put(INCLUDE, Collections.singletonList("children.friends")); + testScope.setQueryParams(queryParams); includedProcessor.execute(jsonApiDocument, parentRecord1, Optional.of(queryParams)); List expectedIncluded = @@ -170,6 +165,7 @@ public void testExecuteMultipleRelations() throws Exception { MultivaluedMap queryParams = new MultivaluedHashMap<>(); queryParams.put(INCLUDE, Arrays.asList("children", "spouses")); + testScope.setQueryParams(queryParams); includedProcessor.execute(jsonApiDocument, parentRecord1, Optional.of(queryParams)); List expectedIncluded = @@ -186,6 +182,7 @@ public void testExecuteMultipleNestedRelations() throws Exception { MultivaluedMap queryParams = new MultivaluedHashMap<>(); queryParams.put(INCLUDE, Collections.singletonList("children.friends")); + testScope.setQueryParams(queryParams); includedProcessor.execute(jsonApiDocument, parentRecord3, Optional.of(queryParams)); Set expectedIncluded = @@ -207,6 +204,7 @@ public void testIncludeForbiddenRelationship() { MultivaluedMap queryParams = new MultivaluedHashMap<>(); queryParams.put(INCLUDE, Collections.singletonList("relation1")); + testScope.setQueryParams(queryParams); includedProcessor.execute(jsonApiDocument, funWithPermissionsRecord, Optional.of(queryParams)); assertNull(jsonApiDocument.getIncluded(), @@ -229,6 +227,7 @@ public void testNoQueryIncludeParams() throws Exception { MultivaluedMap queryParams = new MultivaluedHashMap<>(); queryParams.put("unused", Collections.emptyList()); + testScope.setQueryParams(queryParams); includedProcessor.execute(jsonApiDocument, parentRecord1, Optional.of(queryParams)); assertNull(jsonApiDocument.getIncluded(), diff --git a/elide-core/src/test/java/com/yahoo/elide/parsers/expression/CanPaginateVisitorTest.java b/elide-core/src/test/java/com/yahoo/elide/parsers/expression/CanPaginateVisitorTest.java index e620771243..1ba0e7a472 100644 --- a/elide-core/src/test/java/com/yahoo/elide/parsers/expression/CanPaginateVisitorTest.java +++ b/elide-core/src/test/java/com/yahoo/elide/parsers/expression/CanPaginateVisitorTest.java @@ -15,6 +15,7 @@ import com.yahoo.elide.annotation.ReadPermission; import com.yahoo.elide.core.EntityDictionary; import com.yahoo.elide.core.RequestScope; +import com.yahoo.elide.core.TestDictionary; import com.yahoo.elide.core.filter.expression.FilterExpression; import com.yahoo.elide.security.ChangeSpec; import com.yahoo.elide.security.FilterExpressionCheck; @@ -90,7 +91,7 @@ class Book { private String title; } - EntityDictionary dictionary = new EntityDictionary(checkMappings); + EntityDictionary dictionary = TestDictionary.getTestDictionary(checkMappings); dictionary.bindEntity(Book.class); RequestScope scope = mock(RequestScope.class); @@ -109,7 +110,7 @@ class Book { private String title; } - EntityDictionary dictionary = new EntityDictionary(checkMappings); + EntityDictionary dictionary = TestDictionary.getTestDictionary(checkMappings); dictionary.bindEntity(Book.class); RequestScope scope = mock(RequestScope.class); @@ -127,7 +128,7 @@ class Book { private String title; } - EntityDictionary dictionary = new EntityDictionary(checkMappings); + EntityDictionary dictionary = TestDictionary.getTestDictionary(checkMappings); dictionary.bindEntity(Book.class); RequestScope scope = mock(RequestScope.class); @@ -146,7 +147,7 @@ class Book { private String title; } - EntityDictionary dictionary = new EntityDictionary(checkMappings); + EntityDictionary dictionary = TestDictionary.getTestDictionary(checkMappings); dictionary.bindEntity(Book.class); RequestScope scope = mock(RequestScope.class); @@ -166,7 +167,7 @@ class Book { private String title; } - EntityDictionary dictionary = new EntityDictionary(checkMappings); + EntityDictionary dictionary = TestDictionary.getTestDictionary(checkMappings); dictionary.bindEntity(Book.class); RequestScope scope = mock(RequestScope.class); @@ -185,7 +186,7 @@ class Book { private String title; } - EntityDictionary dictionary = new EntityDictionary(checkMappings); + EntityDictionary dictionary = TestDictionary.getTestDictionary(checkMappings); dictionary.bindEntity(Book.class); RequestScope scope = mock(RequestScope.class); @@ -204,7 +205,7 @@ class Book { private String title; } - EntityDictionary dictionary = new EntityDictionary(checkMappings); + EntityDictionary dictionary = TestDictionary.getTestDictionary(checkMappings); dictionary.bindEntity(Book.class); RequestScope scope = mock(RequestScope.class); @@ -223,7 +224,7 @@ class Book { private String title; } - EntityDictionary dictionary = new EntityDictionary(checkMappings); + EntityDictionary dictionary = TestDictionary.getTestDictionary(checkMappings); dictionary.bindEntity(Book.class); RequestScope scope = mock(RequestScope.class); @@ -242,7 +243,7 @@ class Book { private String title; } - EntityDictionary dictionary = new EntityDictionary(checkMappings); + EntityDictionary dictionary = TestDictionary.getTestDictionary(checkMappings); dictionary.bindEntity(Book.class); RequestScope scope = mock(RequestScope.class); @@ -261,7 +262,7 @@ class Book { private String title; } - EntityDictionary dictionary = new EntityDictionary(checkMappings); + EntityDictionary dictionary = TestDictionary.getTestDictionary(checkMappings); dictionary.bindEntity(Book.class); RequestScope scope = mock(RequestScope.class); @@ -283,7 +284,7 @@ class Book { private Date publicationDate; } - EntityDictionary dictionary = new EntityDictionary(checkMappings); + EntityDictionary dictionary = TestDictionary.getTestDictionary(checkMappings); dictionary.bindEntity(Book.class); RequestScope scope = mock(RequestScope.class); @@ -306,7 +307,7 @@ class Book { private Date publicationDate; } - EntityDictionary dictionary = new EntityDictionary(checkMappings); + EntityDictionary dictionary = TestDictionary.getTestDictionary(checkMappings); dictionary.bindEntity(Book.class); RequestScope scope = mock(RequestScope.class); @@ -331,7 +332,7 @@ class Book { private boolean outOfPrint; } - EntityDictionary dictionary = new EntityDictionary(checkMappings); + EntityDictionary dictionary = TestDictionary.getTestDictionary(checkMappings); dictionary.bindEntity(Book.class); RequestScope scope = mock(RequestScope.class); diff --git a/elide-core/src/test/java/com/yahoo/elide/parsers/expression/PermissionExpressionVisitorTest.java b/elide-core/src/test/java/com/yahoo/elide/parsers/expression/PermissionExpressionVisitorTest.java index 5c463ba470..c83f8b28e3 100644 --- a/elide-core/src/test/java/com/yahoo/elide/parsers/expression/PermissionExpressionVisitorTest.java +++ b/elide-core/src/test/java/com/yahoo/elide/parsers/expression/PermissionExpressionVisitorTest.java @@ -13,6 +13,7 @@ import com.yahoo.elide.annotation.ReadPermission; import com.yahoo.elide.annotation.UpdatePermission; import com.yahoo.elide.core.EntityDictionary; +import com.yahoo.elide.core.TestDictionary; import com.yahoo.elide.security.ChangeSpec; import com.yahoo.elide.security.RequestScope; import com.yahoo.elide.security.checks.Check; @@ -47,7 +48,7 @@ public void setupEntityDictionary() { checks.put("user has all access", Role.ALL.class); checks.put("user has no access", Role.NONE.class); - dictionary = new EntityDictionary(checks); + dictionary = TestDictionary.getTestDictionary(checks); dictionary.bindEntity(Model.class); dictionary.bindEntity(ComplexEntity.class); } diff --git a/elide-core/src/test/java/com/yahoo/elide/parsers/expression/PermissionToFilterExpressionVisitorTest.java b/elide-core/src/test/java/com/yahoo/elide/parsers/expression/PermissionToFilterExpressionVisitorTest.java index 310b9531c5..ee97249e46 100644 --- a/elide-core/src/test/java/com/yahoo/elide/parsers/expression/PermissionToFilterExpressionVisitorTest.java +++ b/elide-core/src/test/java/com/yahoo/elide/parsers/expression/PermissionToFilterExpressionVisitorTest.java @@ -19,6 +19,7 @@ import com.yahoo.elide.core.EntityPermissions; import com.yahoo.elide.core.Path; import com.yahoo.elide.core.RequestScope; +import com.yahoo.elide.core.TestDictionary; import com.yahoo.elide.core.filter.FilterPredicate; import com.yahoo.elide.core.filter.Operator; import com.yahoo.elide.core.filter.expression.AndFilterExpression; @@ -94,7 +95,7 @@ public void setupEntityDictionary() { checks.put(LT_FILTER, Permissions.LessThanFilterExpression.class); checks.put(GE_FILTER, Permissions.GreaterThanOrEqualFilterExpression.class); - dictionary = new EntityDictionary(checks); + dictionary = TestDictionary.getTestDictionary(checks); elideSettings = new ElideSettingsBuilder(null) .withEntityDictionary(dictionary) .build(); diff --git a/elide-core/src/test/java/com/yahoo/elide/security/PermissionExecutorTest.java b/elide-core/src/test/java/com/yahoo/elide/security/PermissionExecutorTest.java index 9777d095c8..b037737c8a 100644 --- a/elide-core/src/test/java/com/yahoo/elide/security/PermissionExecutorTest.java +++ b/elide-core/src/test/java/com/yahoo/elide/security/PermissionExecutorTest.java @@ -17,11 +17,11 @@ import com.yahoo.elide.core.EntityDictionary; import com.yahoo.elide.core.PersistentResource; import com.yahoo.elide.core.RequestScope; +import com.yahoo.elide.core.TestDictionary; import com.yahoo.elide.core.exceptions.ForbiddenAccessException; import com.yahoo.elide.security.checks.CommitCheck; import com.yahoo.elide.security.checks.OperationCheck; import com.yahoo.elide.security.checks.UserCheck; - import com.yahoo.elide.security.permissions.ExpressionResult; import example.TestCheckMappings; @@ -456,8 +456,8 @@ public PersistentResource newResource(T obj, Class cls) { return new PersistentResource<>(obj, null, requestScope.getUUIDFor(obj), requestScope); } - public PersistentResource newResource(Class cls) { - EntityDictionary dictionary = new EntityDictionary(TestCheckMappings.MAPPINGS); + public PersistentResource newResource(Class cls) { + EntityDictionary dictionary = TestDictionary.getTestDictionary(); dictionary.bindEntity(cls); RequestScope requestScope = new RequestScope(null, null, null, null, null, getElideSettings(dictionary)); try { diff --git a/elide-core/src/test/java/com/yahoo/elide/security/permissions/PermissionExpressionBuilderTest.java b/elide-core/src/test/java/com/yahoo/elide/security/permissions/PermissionExpressionBuilderTest.java index e09a757a15..2b1df5c2c2 100644 --- a/elide-core/src/test/java/com/yahoo/elide/security/permissions/PermissionExpressionBuilderTest.java +++ b/elide-core/src/test/java/com/yahoo/elide/security/permissions/PermissionExpressionBuilderTest.java @@ -15,6 +15,7 @@ import com.yahoo.elide.core.EntityDictionary; import com.yahoo.elide.core.PersistentResource; import com.yahoo.elide.core.RequestScope; +import com.yahoo.elide.core.TestDictionary; import com.yahoo.elide.security.ChangeSpec; import com.yahoo.elide.security.checks.Check; import com.yahoo.elide.security.checks.prefab.Role; @@ -40,7 +41,7 @@ public void setupEntityDictionary() { checks.put("user has all access", Role.ALL.class); checks.put("user has no access", Role.NONE.class); - dictionary = new EntityDictionary(checks); + dictionary = TestDictionary.getTestDictionary(checks); ExpressionResultCache cache = new ExpressionResultCache(); builder = new PermissionExpressionBuilder(cache, dictionary); diff --git a/elide-datastore/elide-datastore-aggregation/.gitignore b/elide-datastore/elide-datastore-aggregation/.gitignore new file mode 100644 index 0000000000..2f7896d1d1 --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/.gitignore @@ -0,0 +1 @@ +target/ diff --git a/elide-datastore/elide-datastore-aggregation/pom.xml b/elide-datastore/elide-datastore-aggregation/pom.xml new file mode 100644 index 0000000000..c2da5362a9 --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/pom.xml @@ -0,0 +1,203 @@ + + + + 4.0.0 + elide-datastore-aggregation + jar + Elide Data Store: Aggregation Data Store + Elide Data Store for Aggregation + https://github.com/yahoo/elide/tree/master/elide-datastore/elide-datastore-aggregation + + com.yahoo.elide + elide-datastore-parent-pom + 5.0.0-pr6-SNAPSHOT + + + + + The Apache Software License, Version 2.0 + http://www.apache.org/licenses/LICENSE-2.0.txt + repo + + + + + + Yahoo Inc. + https://github.com/yahoo + + + + + scm:git:ssh://git@github.com/yahoo/elide.git + https://github.com/yahoo/elide.git + HEAD + + + + 5.4.1 + + + + + com.yahoo.elide + elide-core + 5.0.0-pr6-SNAPSHOT + + + + com.yahoo.elide + elide-datastore-jpa + 5.0.0-pr6-SNAPSHOT + + + + com.yahoo.elide + elide-graphql + 5.0.0-pr6-SNAPSHOT + + + com.yahoo.elide + elide-datastore-multiplex + 5.0.0-pr6-SNAPSHOT + + + + org.projectlombok + lombok + + + + + org.hibernate.javax.persistence + hibernate-jpa-2.1-api + 1.0.0.Final + + + + + org.hibernate + hibernate-entitymanager + ${hibernate5.version} + + + + + + org.junit.jupiter + junit-jupiter-api + ${junit.version} + test + + + + org.junit.jupiter + junit-jupiter-params + ${junit.version} + test + + + + org.junit.jupiter + junit-jupiter-engine + ${junit.version} + test + + + + + com.h2database + h2 + 1.4.197 + test + + + + + org.mockito + mockito-core + test + + + + + com.yahoo.elide + elide-integration-tests + 5.0.0-pr6-SNAPSHOT + test + test-jar + + + + + org.eclipse.jetty + jetty-servlet + test + + + + org.eclipse.jetty + jetty-webapp + test + + + + + org.glassfish.jersey.containers + jersey-container-servlet-core + 2.29.1 + test + + + + org.glassfish.jersey.inject + jersey-hk2 + test + + + + org.glassfish.jersey.containers + jersey-container-servlet + test + + + + io.rest-assured + rest-assured + 4.0.0 + test + + + + org.glassfish.hk2 + hk2-api + 2.5.0 + test + + + + + + + org.codehaus.mojo + build-helper-maven-plugin + + + org.apache.maven.plugins + maven-dependency-plugin + + + org.apache.maven.plugins + maven-surefire-plugin + + + org.apache.maven.plugins + maven-checkstyle-plugin + + + + diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/AggregationDataStore.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/AggregationDataStore.java new file mode 100644 index 0000000000..b32a42dd14 --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/AggregationDataStore.java @@ -0,0 +1,72 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation; + +import com.yahoo.elide.core.ArgumentType; +import com.yahoo.elide.core.DataStore; +import com.yahoo.elide.core.DataStoreTransaction; +import com.yahoo.elide.core.EntityDictionary; +import com.yahoo.elide.datastores.aggregation.metadata.MetaDataStore; +import com.yahoo.elide.datastores.aggregation.metadata.models.AnalyticView; +import com.yahoo.elide.datastores.aggregation.metadata.models.TimeDimension; +import com.yahoo.elide.datastores.aggregation.queryengines.sql.annotation.FromSubquery; +import com.yahoo.elide.datastores.aggregation.queryengines.sql.annotation.FromTable; +import com.yahoo.elide.utils.ClassScanner; + +import java.lang.annotation.Annotation; + +/** + * DataStore that supports Aggregation. Uses {@link QueryEngine} to return results. + */ +public class AggregationDataStore implements DataStore { + + private final QueryEngineFactory queryEngineFactory; + + private final MetaDataStore metaDataStore; + + private QueryEngine queryEngine; + + /** + * These are the classes the Aggregation Store manages. + */ + private static final Class[] AGGREGATION_STORE_CLASSES = { + FromTable.class, FromSubquery.class }; + + public AggregationDataStore(QueryEngineFactory queryEngineFactory, + MetaDataStore metaDataStore) { + this.queryEngineFactory = queryEngineFactory; + this.metaDataStore = metaDataStore; + } + + /** + * Populate an {@link EntityDictionary} and use this dictionary to construct a {@link QueryEngine}. + * @param dictionary the dictionary + */ + @Override + public void populateEntityDictionary(EntityDictionary dictionary) { + for (Class cls : AGGREGATION_STORE_CLASSES) { + // bind non-jpa entities, including analyticViews and views + ClassScanner.getAnnotatedClasses(cls).forEach(dictionary::bindEntity); + } + + queryEngine = queryEngineFactory.buildQueryEngine(metaDataStore); + + /* Add 'grain' argument to each TimeDimensionColumn */ + for (AnalyticView table : metaDataStore.getMetaData(AnalyticView.class)) { + for (TimeDimension timeDim : table.getColumns(TimeDimension.class)) { + dictionary.addArgumentToAttribute( + table.getCls(), + timeDim.getName(), + new ArgumentType("grain", String.class)); + } + } + } + + @Override + public DataStoreTransaction beginTransaction() { + return new AggregationDataStoreTransaction(queryEngine); + } +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/AggregationDataStoreTransaction.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/AggregationDataStoreTransaction.java new file mode 100644 index 0000000000..fd6daed82b --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/AggregationDataStoreTransaction.java @@ -0,0 +1,71 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation; + +import com.yahoo.elide.core.DataStoreTransaction; +import com.yahoo.elide.core.RequestScope; +import com.yahoo.elide.datastores.aggregation.metadata.models.Table; +import com.yahoo.elide.datastores.aggregation.query.Query; +import com.yahoo.elide.request.EntityProjection; + +import com.google.common.annotations.VisibleForTesting; + +import java.io.IOException; + +/** + * Transaction handler for {@link AggregationDataStore}. + */ +public class AggregationDataStoreTransaction implements DataStoreTransaction { + private QueryEngine queryEngine; + + public AggregationDataStoreTransaction(QueryEngine queryEngine) { + this.queryEngine = queryEngine; + } + + @Override + public void save(Object entity, RequestScope scope) { + + } + + @Override + public void delete(Object entity, RequestScope scope) { + + } + + @Override + public void flush(RequestScope scope) { + + } + + @Override + public void commit(RequestScope scope) { + + } + + @Override + public void createObject(Object entity, RequestScope scope) { + + } + + @Override + public Iterable loadObjects(EntityProjection entityProjection, RequestScope scope) { + Query query = buildQuery(entityProjection, scope); + return queryEngine.executeQuery(query); + } + + @Override + public void close() throws IOException { + + } + + @VisibleForTesting + private Query buildQuery(EntityProjection entityProjection, RequestScope scope) { + Table table = queryEngine.getTable(entityProjection.getType()); + EntityProjectionTranslator translator = new EntityProjectionTranslator(table, + entityProjection, scope.getDictionary()); + return translator.getQuery(); + } +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/EntityProjectionTranslator.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/EntityProjectionTranslator.java new file mode 100644 index 0000000000..86ed3dfd7b --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/EntityProjectionTranslator.java @@ -0,0 +1,206 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation; + +import com.yahoo.elide.core.EntityDictionary; +import com.yahoo.elide.core.exceptions.InvalidOperationException; +import com.yahoo.elide.core.filter.expression.FilterExpression; +import com.yahoo.elide.datastores.aggregation.filter.visitor.FilterConstraints; +import com.yahoo.elide.datastores.aggregation.filter.visitor.SplitFilterExpressionVisitor; +import com.yahoo.elide.datastores.aggregation.metadata.metric.MetricFunctionInvocation; +import com.yahoo.elide.datastores.aggregation.metadata.models.AnalyticView; +import com.yahoo.elide.datastores.aggregation.metadata.models.Dimension; +import com.yahoo.elide.datastores.aggregation.metadata.models.Table; +import com.yahoo.elide.datastores.aggregation.metadata.models.TimeDimension; +import com.yahoo.elide.datastores.aggregation.metadata.models.TimeDimensionGrain; +import com.yahoo.elide.datastores.aggregation.query.ColumnProjection; +import com.yahoo.elide.datastores.aggregation.query.Query; +import com.yahoo.elide.datastores.aggregation.query.TimeDimensionProjection; +import com.yahoo.elide.request.Argument; +import com.yahoo.elide.request.Attribute; +import com.yahoo.elide.request.EntityProjection; +import com.yahoo.elide.request.Relationship; + +import com.google.common.collect.Sets; + +import java.util.LinkedHashSet; +import java.util.List; +import java.util.Locale; +import java.util.Objects; +import java.util.Set; +import java.util.stream.Collectors; + +/** + * Helper for Aggregation Data Store which does the work associated with extracting {@link Query}. + */ +public class EntityProjectionTranslator { + private AnalyticView queriedTable; + + private EntityProjection entityProjection; + private Set dimensionProjections; + private Set timeDimensions; + private List metrics; + private FilterExpression whereFilter; + private FilterExpression havingFilter; + private EntityDictionary dictionary; + + public EntityProjectionTranslator(Table table, EntityProjection entityProjection, EntityDictionary dictionary) { + if (!(table instanceof AnalyticView)) { + throw new InvalidOperationException("Queried table is not analyticView: " + table.getName()); + } + + this.queriedTable = (AnalyticView) table; + this.entityProjection = entityProjection; + this.dictionary = dictionary; + dimensionProjections = resolveNonTimeDimensions(); + timeDimensions = resolveTimeDimensions(); + metrics = resolveMetrics(); + splitFilters(); + } + + /** + * Builds the query from internal state. + * @return {@link Query} query object with all the parameters provided by user. + */ + public Query getQuery() { + Query query = Query.builder() + .analyticView(queriedTable) + .metrics(metrics) + .groupByDimensions(dimensionProjections) + .timeDimensions(timeDimensions) + .whereFilter(whereFilter) + .havingFilter(havingFilter) + .sorting(entityProjection.getSorting()) + .pagination(entityProjection.getPagination()) + .build(); + QueryValidator validator = new QueryValidator(query, getAllFields(), dictionary); + validator.validate(); + return query; + } + + /** + * Gets whereFilter and havingFilter based on provided filter expression from {@link EntityProjection}. + */ + private void splitFilters() { + FilterExpression filterExpression = entityProjection.getFilterExpression(); + if (filterExpression == null) { + whereFilter = null; + havingFilter = null; + return; + } + SplitFilterExpressionVisitor visitor = new SplitFilterExpressionVisitor(queriedTable); + FilterConstraints constraints = filterExpression.accept(visitor); + whereFilter = constraints.getWhereExpression(); + havingFilter = constraints.getHavingExpression(); + } + + /** + * Gets time dimensions based on relationships and attributes from {@link EntityProjection}. + * + * @return projections for time dimension columns + * @throws InvalidOperationException Thrown if a requested time grain is not supported. + */ + private Set resolveTimeDimensions() { + return entityProjection.getAttributes().stream() + .filter(attribute -> queriedTable.getTimeDimension(attribute.getName()) != null) + .map(timeDimAttr -> { + TimeDimension timeDim = queriedTable.getTimeDimension(timeDimAttr.getName()); + + Argument grainArgument = timeDimAttr.getArguments().stream() + .filter(attr -> attr.getName().equals("grain")) + .findAny() + .orElse(null); + + TimeDimensionGrain resolvedGrain; + if (grainArgument == null) { + //The first grain is the default. + resolvedGrain = timeDim.getSupportedGrains().stream() + .findFirst() + .orElseThrow(() -> new IllegalStateException( + String.format("Requested default grain, no grain defined on %s", + timeDimAttr.getName()))); + } else { + String requestedGrainName = grainArgument.getValue().toString(); + + resolvedGrain = timeDim.getSupportedGrains().stream() + .filter(supportedGrain -> supportedGrain.getGrain().name().toLowerCase(Locale.ENGLISH) + .equals(requestedGrainName)) + .findFirst() + .orElseThrow(() -> new InvalidOperationException( + String.format("Unsupported grain %s for field %s", + requestedGrainName, + timeDimAttr.getName()))); + } + + return ColumnProjection.toProjection(timeDim, resolvedGrain.getGrain(), timeDimAttr.getAlias()); + }) + .collect(Collectors.toCollection(LinkedHashSet::new)); + } + + /** + * Gets dimensions except time dimensions based on relationships and attributes from {@link EntityProjection}. + */ + private Set resolveNonTimeDimensions() { + Set attributes = entityProjection.getAttributes().stream() + .filter(attribute -> queriedTable.getTimeDimension(attribute.getName()) == null) + .map(dimAttr -> { + Dimension dimension = queriedTable.getDimension(dimAttr.getName()); + return dimension == null ? null : ColumnProjection.toProjection(dimension, dimAttr.getAlias()); + }) + .filter(Objects::nonNull) + .collect(Collectors.toSet()); + + Set relationships = entityProjection.getRelationships().stream() + .map(dimAttr -> { + Dimension dimension = queriedTable.getDimension(dimAttr.getName()); + return dimension == null ? null : ColumnProjection.toProjection(dimension, dimAttr.getAlias()); + }) + .filter(Objects::nonNull) + .collect(Collectors.toSet()); + + return Sets.union(attributes, relationships); + } + + /** + * Gets metrics based on attributes from {@link EntityProjection}. + */ + private List resolveMetrics() { + return entityProjection.getAttributes().stream() + .filter(attribute -> queriedTable.isMetric(attribute.getName())) + .map(attribute -> queriedTable.getMetric(attribute.getName()) + .getMetricFunction() + .invoke(attribute.getArguments(), attribute.getAlias())) + .collect(Collectors.toList()); + } + + /** + * Gets relationship names from {@link EntityProjection}. + * @return relationships list of {@link Relationship} names + */ + private Set getRelationships() { + return entityProjection.getRelationships().stream() + .map(Relationship::getName).collect(Collectors.toCollection(LinkedHashSet::new)); + } + + /** + * Gets attribute names from {@link EntityProjection}. + * @return relationships list of {@link Attribute} names + */ + private Set getAttributes() { + return entityProjection.getAttributes().stream() + .map(Attribute::getName).collect(Collectors.toCollection(LinkedHashSet::new)); + } + + /** + * Helper method to get all field names from the {@link EntityProjection}. + * @return allFields set of all field names + */ + private Set getAllFields() { + Set allFields = getAttributes(); + allFields.addAll(getRelationships()); + return allFields; + } +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/QueryEngine.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/QueryEngine.java new file mode 100644 index 0000000000..74dc06a008 --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/QueryEngine.java @@ -0,0 +1,72 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation; + +import com.yahoo.elide.core.DataStore; +import com.yahoo.elide.core.DataStoreTransaction; +import com.yahoo.elide.datastores.aggregation.metadata.models.Table; +import com.yahoo.elide.datastores.aggregation.query.Query; + +/** + * A {@link QueryEngine} is an abstraction that an AggregationDataStore leverages to run analytic queries (OLAP style) + * against an underlying persistence layer. + *

+ * The purpose of {@link QueryEngine} is to allow a single {@link DataStore} to utilize multiple query frameworks, such + * as JPA on SQL or NoSQL query engine on Druid shown below. + * 

+ *        +-----------+
+ *        |           |
+ *        | DataStore |
+ *        |           |
+ *        +-----+-----+
+ *              |
+ *              |
+ *   +----------v-----------+
+ *   |                      |
+ *   | DataStoreTransaction |
+ *   |                      |
+ *   +----------+-----------+
+ *              |
+ *              |
+ *       +------v------+
+ *       |             |
+ *       | QueryEngine |
+ *       |             |
+ *       +------+------+
+ *              |
+ *              |
+ *     +--------+---------+
+ *     |                  |
+ * +---v---+          +---v---+
+ * |       |          |       |
+ * | Druid |          | MySQL |
+ * |       |          |       |
+ * +-------+          +-------+
+ *
+ * Implementor must assume that {@link DataStoreTransaction} will never keep reference to any internal state of a + * {@link QueryEngine} object. This ensures the plugability of various {@link QueryEngine} implementations. + *

+ * This is a {@link java.util.function functional interface} whose functional method is {@link #executeQuery(Query)}. + */ +public interface QueryEngine { + + /** + * Executes the specified {@link Query} against a specific persistent storage, which understand the provided + * {@link Query}. + * + * @param query The query customized for a particular persistent storage or storage client + * + * @return query results + */ + Iterable executeQuery(Query query); + + /** + * Returns the schema for a given entity class. + * @param entityClass The class to map to a schema. + * @return The schema that represents the provided entity. + */ + Table getTable(Class entityClass); +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/QueryEngineFactory.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/QueryEngineFactory.java new file mode 100644 index 0000000000..7a77dfdf6d --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/QueryEngineFactory.java @@ -0,0 +1,15 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation; + +import com.yahoo.elide.datastores.aggregation.metadata.MetaDataStore; + +/** + * Interface that constructs {@link QueryEngine} based on given entityDictionary. + */ +public interface QueryEngineFactory { + QueryEngine buildQueryEngine(MetaDataStore metaDataStore); +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/QueryValidator.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/QueryValidator.java new file mode 100644 index 0000000000..a6ad1583fe --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/QueryValidator.java @@ -0,0 +1,155 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation; + +import com.yahoo.elide.core.EntityDictionary; +import com.yahoo.elide.core.Path; +import com.yahoo.elide.core.exceptions.InvalidOperationException; +import com.yahoo.elide.core.filter.FilterPredicate; +import com.yahoo.elide.core.filter.expression.AndFilterExpression; +import com.yahoo.elide.core.filter.expression.FilterExpression; +import com.yahoo.elide.core.filter.expression.NotFilterExpression; +import com.yahoo.elide.core.filter.expression.OrFilterExpression; +import com.yahoo.elide.core.sort.Sorting; +import com.yahoo.elide.datastores.aggregation.metadata.metric.MetricFunctionInvocation; +import com.yahoo.elide.datastores.aggregation.metadata.models.AnalyticView; +import com.yahoo.elide.datastores.aggregation.query.ColumnProjection; +import com.yahoo.elide.datastores.aggregation.query.Query; + +import java.util.List; +import java.util.Map; +import java.util.Set; + +/** + * Class that checks whether a constructed {@link Query} object can be executed. + * Checks include validate sorting, having clause and make sure there is at least 1 metric queried. + */ +public class QueryValidator { + private Query query; + private Set allFields; + private EntityDictionary dictionary; + private AnalyticView queriedTable; + private Class queriedClass; + private List metrics; + private Set dimensionProjections; + + public QueryValidator(Query query, Set allFields, EntityDictionary dictionary) { + this.query = query; + this.allFields = allFields; + this.dictionary = dictionary; + this.queriedTable = query.getAnalyticView(); + this.queriedClass = queriedTable.getCls(); + this.metrics = query.getMetrics(); + this.dimensionProjections = query.getDimensions(); + } + + /** + * Method that handles all checks to make sure query is valid before we attempt to execute the query. + */ + public void validate() { + validateHavingClause(query.getHavingFilter()); + validateSorting(); + } + + /** + * Validate the having clause before execution. Having clause is not as flexible as where clause, + * the fields in having clause must be either or these two: + * 1. A grouped by dimension in this query + * 2. An aggregated metric in this query + * + * All grouped by dimensions are defined in the entity bean, so the last entity class of a filter path + * must match entity class of the query. + * + * @param havingClause having clause generated from this query + */ + private void validateHavingClause(FilterExpression havingClause) { + // TODO: support having clause for alias + if (havingClause instanceof FilterPredicate) { + Path path = ((FilterPredicate) havingClause).getPath(); + Path.PathElement last = path.lastElement().get(); + Class cls = last.getType(); + String fieldName = last.getFieldName(); + + if (cls != queriedTable.getCls()) { + throw new InvalidOperationException( + String.format( + "Can't filter on relationship field %s in HAVING clause when querying table %s.", + path.toString(), + queriedTable.getCls().getSimpleName())); + } + + if (queriedTable.isMetric(fieldName)) { + if (metrics.stream().noneMatch(m -> m.getAlias().equals(fieldName))) { + throw new InvalidOperationException( + String.format( + "Metric field %s must be aggregated before filtering in having clause.", + fieldName)); + } + } else { + if (dimensionProjections.stream().noneMatch(dim -> dim.getAlias().equals(fieldName))) { + throw new InvalidOperationException( + String.format( + "Dimension field %s must be grouped before filtering in having clause.", + fieldName)); + } + } + } else if (havingClause instanceof AndFilterExpression) { + validateHavingClause(((AndFilterExpression) havingClause).getLeft()); + validateHavingClause(((AndFilterExpression) havingClause).getRight()); + } else if (havingClause instanceof OrFilterExpression) { + validateHavingClause(((OrFilterExpression) havingClause).getLeft()); + validateHavingClause(((OrFilterExpression) havingClause).getRight()); + } else if (havingClause instanceof NotFilterExpression) { + validateHavingClause(((NotFilterExpression) havingClause).getNegated()); + } + } + + /** + * Method to verify that all the sorting options provided + * by the user are valid and supported. + */ + public void validateSorting() { + Sorting sorting = query.getSorting(); + if (sorting == null) { + return; + } + Map sortClauses = sorting.getValidSortingRules(queriedClass, dictionary); + sortClauses.keySet().forEach((path) -> validateSortingPath(path, allFields)); + } + + /** + * Verifies that the current path can be sorted on + * @param path The path that we are validating + * @param allFields Set of all field names included in initial query + */ + private void validateSortingPath(Path path, Set allFields) { + List pathElements = path.getPathElements(); + + // TODO: add support for double nested sorting + if (pathElements.size() > 2) { + throw new UnsupportedOperationException( + "Currently sorting on double nested fields is not supported"); + } + + if (metrics.isEmpty() && pathElements.size() > 1) { + throw new UnsupportedOperationException( + "Query with no metric can't sort on nested field."); + } + + Path.PathElement currentElement = pathElements.get(0); + String currentField = currentElement.getFieldName(); + Class currentClass = currentElement.getType(); + + // TODO: support sorting using alias + if (allFields.stream().noneMatch(field -> field.equals(currentField))) { + throw new InvalidOperationException("Can't sort on " + currentField + " as it is not present in query"); + } + if (dictionary.getIdFieldName(currentClass).equals(currentField) + || currentField.equals(EntityDictionary.REGULAR_ID_NAME)) { + throw new InvalidOperationException("Sorting on id field is not permitted"); + } + } +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/annotation/Cardinality.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/annotation/Cardinality.java new file mode 100644 index 0000000000..3be3e35faf --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/annotation/Cardinality.java @@ -0,0 +1,44 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.annotation; + +import static java.lang.annotation.ElementType.FIELD; +import static java.lang.annotation.ElementType.METHOD; +import static java.lang.annotation.ElementType.TYPE; + +import java.lang.annotation.Documented; +import java.lang.annotation.ElementType; +import java.lang.annotation.Retention; +import java.lang.annotation.RetentionPolicy; +import java.lang.annotation.Target; + +/** + * Indicates whether a dimension has small, medium, or large cardinality. + *

+ * Example: {@literal @}Cardinality(size = {@link CardinalitySize#MEDIUM}). If {@code size} is not specified, + * {@link CardinalitySize#LARGE} is used by default. See {@link CardinalitySize}. + *

+ * In the case of double binding, the following precedence rule is applied: + *

    + *
  1. {@link ElementType#TYPE} + *
  2. {@link ElementType#METHOD} or {@link ElementType#FIELD} + *
+ */ +@Documented +@Target({ TYPE, FIELD, METHOD }) +@Retention(RetentionPolicy.RUNTIME) +public @interface Cardinality { + + /** + * Returns the size category of a dimension. + *

+ * The size category must be from one of the values of type {@link CardinalitySize}. {@link CardinalitySize#LARGE} + * will be the default if size is not specified. + * + * @return dimension size + */ + CardinalitySize size() default CardinalitySize.LARGE; +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/annotation/CardinalitySize.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/annotation/CardinalitySize.java new file mode 100644 index 0000000000..13766762df --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/annotation/CardinalitySize.java @@ -0,0 +1,34 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.annotation; + +/** + * A set of constants that indicates how big a dimension is. + */ +public enum CardinalitySize { + + /** + * Size for a small dimension table. + *

+ * TODO: define size range + */ + SMALL, + + /** + * Size for a medium sized dimension table. + *

+ * TODO: define size range + */ + MEDIUM, + + /** + * Size for a large dimension table. + *

+ * TODO: define size range + */ + LARGE + ; +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/annotation/FriendlyName.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/annotation/FriendlyName.java new file mode 100644 index 0000000000..ae8e06597d --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/annotation/FriendlyName.java @@ -0,0 +1,23 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.annotation; + +import java.lang.annotation.Documented; +import java.lang.annotation.ElementType; +import java.lang.annotation.Retention; +import java.lang.annotation.RetentionPolicy; +import java.lang.annotation.Target; + +/** + * Indicates that an annotated field is a friendlyName or human displayable column for that dimension. + */ +@Documented +@Target({ElementType.FIELD, ElementType.METHOD}) +@Retention(RetentionPolicy.RUNTIME) +public @interface FriendlyName { + + // intentionally left blank +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/annotation/Meta.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/annotation/Meta.java new file mode 100644 index 0000000000..384eb4abd8 --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/annotation/Meta.java @@ -0,0 +1,25 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.annotation; + +import java.lang.annotation.Documented; +import java.lang.annotation.ElementType; +import java.lang.annotation.Retention; +import java.lang.annotation.RetentionPolicy; +import java.lang.annotation.Target; + +/** + * Indicates that the specified entity field has a configured long name and field description for human to read on UI. + */ +@Documented +@Target({ElementType.FIELD, ElementType.METHOD, ElementType.TYPE}) +@Retention(RetentionPolicy.RUNTIME) +public @interface Meta { + + String longName() default ""; + + String description() default ""; +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/annotation/MetricAggregation.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/annotation/MetricAggregation.java new file mode 100644 index 0000000000..73da16f13c --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/annotation/MetricAggregation.java @@ -0,0 +1,24 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.annotation; + +import com.yahoo.elide.datastores.aggregation.metadata.models.MetricFunction; + +import java.lang.annotation.Documented; +import java.lang.annotation.ElementType; +import java.lang.annotation.Retention; +import java.lang.annotation.RetentionPolicy; +import java.lang.annotation.Target; + +/** + * Specify that a field in a table is metric field. + */ +@Documented +@Target({ElementType.FIELD, ElementType.METHOD}) +@Retention(RetentionPolicy.RUNTIME) +public @interface MetricAggregation { + Class function(); +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/annotation/MetricComputation.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/annotation/MetricComputation.java new file mode 100644 index 0000000000..8762d856b3 --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/annotation/MetricComputation.java @@ -0,0 +1,62 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.annotation; + +import com.yahoo.elide.datastores.aggregation.metadata.models.Table; + +import java.lang.annotation.Documented; +import java.lang.annotation.ElementType; +import java.lang.annotation.Retention; +import java.lang.annotation.RetentionPolicy; +import java.lang.annotation.Target; + +/** + * Indicates that a field is computed via a {@link #expression() custom metric formula expression}, such as Calcite SQL. + *

+ * Example: {@literal @}ComputedMetric(expression = '(fieldA * fieldB) / 100').The field names should match the Elide + * data model field names. + *

+ * {@code expression} can also be composite. During {@link Table} construction, it will substitute attribute names in + * the provided expression with either: + *

    + *
  • The column alias for that column in the query, or + *
  • Another ComputedMetric expression - recursively expanding expressions until every referenced field is not + * computed. + *
+ * For example, considering the following entity: + * 
+ * {@code
+ * public class FactTable {
+ *
+ *     {@literal @}MetricAggregation(sum.class)
+ *     Long sessions
+ *
+ *     {@literal @}MetricAggregation(sum.class)
+ *     Long timeSpent
+ *
+ *     {@literal @}MetricComputation(expression = "timeSpent / sessions")
+ *     Float timeSpentPerSession
+ *
+ *     {@literal @}MetricComputation(expression = "timeSpentPerSession / gameCount")
+ *     Float timeSpentPerGame
+ * }
+ * }
+ *
+ * During {@link Table} construction, {@code timeSpentPerSession} the provided expression will be substituted with + * {@code timeSpent / sessions}. + */ +@Documented +@Target({ElementType.FIELD, ElementType.METHOD}) +@Retention(RetentionPolicy.RUNTIME) +public @interface MetricComputation { + + /** + * The custom metric expression that represents this metric computation logic. + * + * @return metric formula + */ + String expression(); +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/annotation/Temporal.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/annotation/Temporal.java new file mode 100644 index 0000000000..0a9e84ea93 --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/annotation/Temporal.java @@ -0,0 +1,47 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.annotation; + +import com.yahoo.elide.datastores.aggregation.time.TimeGrain; + +import java.lang.annotation.Documented; +import java.lang.annotation.ElementType; +import java.lang.annotation.Retention; +import java.lang.annotation.RetentionPolicy; +import java.lang.annotation.Target; +import java.util.TimeZone; + +/** + * Indicates that the annotated entity field is a temporal field and is backed by a temporal column in persistent + * storage. + */ +@Documented +@Target({ElementType.FIELD, ElementType.METHOD}) +@Retention(RetentionPolicy.RUNTIME) +public @interface Temporal { + + /** + * The set of time grains supported by this time dimension. + * + * @return one or more time gains. + */ + TimeGrainDefinition[] grains() default { @TimeGrainDefinition(grain = TimeGrain.DAY, expression = "") }; + + /** + * The timezone in {@link String} of the column. + *

+ * The String format can be expressed by + *

    + *
  • an abbreviation such as "PST", or + *
  • a full name such as "America/Los_Angeles", or + *
  • a custom ID such as "GMT-8:00" + *
+ * The timezone will be parsed using {@link TimeZone#getTimeZone(String)}. + * + * @return data timezone + */ + String timeZone() default "UTC"; +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/annotation/TimeGrainDefinition.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/annotation/TimeGrainDefinition.java new file mode 100644 index 0000000000..6dac86360a --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/annotation/TimeGrainDefinition.java @@ -0,0 +1,29 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.annotation; + +import com.yahoo.elide.datastores.aggregation.time.TimeGrain; + +/** + * A time grain that a time based dimension can be converted to. + */ +public @interface TimeGrainDefinition { + + /** + * The unit into which temporal column can be divided. + * + * @return One of the supported time grains of a persistent storage column + */ + TimeGrain grain() default TimeGrain.DAY; + + /** + * Optional expression used by the QueryEngine to represent the grain natively. + * The value is query engine specific. + * + * @return An expression which defines the grain and is meaningful to the Query Engine. + */ + String expression() default ""; +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/filter/visitor/FilterConstraints.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/filter/visitor/FilterConstraints.java new file mode 100644 index 0000000000..103df56c34 --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/filter/visitor/FilterConstraints.java @@ -0,0 +1,111 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.filter.visitor; + +import com.yahoo.elide.core.filter.expression.FilterExpression; + +import lombok.Getter; + +import java.util.Objects; + +/** + * {@link FilterConstraints} is an auxiliary class for {@link SplitFilterExpressionVisitor} that wraps a {@code WHERE} + * filter expression and {@code HAVING} filter expression. + *

+ * {@link FilterConstraints} is thread-safe and can be accessed by multiple threads. + */ +public class FilterConstraints { + + /** + * Creates a new {@link FilterConstraints} instance that wraps a specified {@code HAVING} filter expression only. + * + * @param havingExpression A pure {@code HAVING} filter expression + * + * @return a new instance of {@link FilterConstraints} + * + * @throws NullPointerException if the provided {@code HAVING} filter expression is {@code null} + */ + public static FilterConstraints pureHaving(FilterExpression havingExpression) { + return new FilterConstraints( + null, + Objects.requireNonNull(havingExpression, "havingExpression") + ); + } + + /** + * Creates a new {@link FilterConstraints} instance that wraps a specified {@code WHERE} filter expression only. + * + * @param whereExpression A pure {@code WHERE} filter expression + * + * @return a new instance of {@link FilterConstraints} + * + * @throws NullPointerException if the provided {@code WHERE} filter expression is {@code null} + */ + public static FilterConstraints pureWhere(FilterExpression whereExpression) { + return new FilterConstraints( + Objects.requireNonNull(whereExpression, "whereExpression"), + null + ); + } + + /** + * Creates a new {@link FilterConstraints} instance that wraps a pair of specified {@code WHERE} filter expression + * and {@code HAVING} filter expression. + * + * @param whereExpression A pure {@code HAVING} filter expression + * @param havingExpression A pure {@code WHERE} filter expression + * + * @return a new instance of {@link FilterConstraints} + * + * @throws NullPointerException if the provided {@code WHERE} or {@code HAVING} filter expression is {@code null} + */ + public static FilterConstraints withWhereAndHaving( + FilterExpression whereExpression, + FilterExpression havingExpression + ) { + return new FilterConstraints( + Objects.requireNonNull(whereExpression, "whereExpression"), + Objects.requireNonNull(havingExpression, "havingExpression") + ); + } + + @Getter + private final FilterExpression whereExpression; + + @Getter + private final FilterExpression havingExpression; + + /** + * Private constructor. + * + * @param whereExpression + * @param havingExpression + */ + private FilterConstraints(FilterExpression whereExpression, FilterExpression havingExpression) { + this.whereExpression = whereExpression; + this.havingExpression = havingExpression; + } + + /** + * Returns whether or not this {@link FilterConstraints} filter expression pair contains only a {@code HAVING} + * expression, i.e. no {@code WHERE} clause. + * + * @return {@code true} if there is {@code HAVING} expression only and not {@code WHERE} expression. + */ + public boolean isPureHaving() { + return getWhereExpression() == null && getHavingExpression() != null; + } + + /** + * Returns whether or not this {@link FilterConstraints} filter expression pair contains only a {@code WHERE} + * expression, i.e. no {@code HAVING} clause. + * + * @return {@code true} if there is {@code HAVING} expression only and not {@code WHERE} expression. + */ + public boolean isPureWhere() { + return getWhereExpression() != null && getHavingExpression() == null; + } +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/filter/visitor/SplitFilterExpressionVisitor.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/filter/visitor/SplitFilterExpressionVisitor.java new file mode 100644 index 0000000000..8ee3a78ec4 --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/filter/visitor/SplitFilterExpressionVisitor.java @@ -0,0 +1,238 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.filter.visitor; + +import com.yahoo.elide.core.filter.FilterPredicate; +import com.yahoo.elide.core.filter.expression.AndFilterExpression; +import com.yahoo.elide.core.filter.expression.FilterExpression; +import com.yahoo.elide.core.filter.expression.FilterExpressionVisitor; +import com.yahoo.elide.core.filter.expression.NotFilterExpression; +import com.yahoo.elide.core.filter.expression.OrFilterExpression; +import com.yahoo.elide.datastores.aggregation.annotation.MetricAggregation; +import com.yahoo.elide.datastores.aggregation.metadata.models.Table; +import com.yahoo.elide.parsers.expression.FilterExpressionNormalizationVisitor; + +import org.apache.commons.lang3.tuple.Pair; + +import lombok.AccessLevel; +import lombok.Getter; +import lombok.extern.slf4j.Slf4j; + +import java.util.Objects; + +/** + * {@link SplitFilterExpressionVisitor} splits the {@link FilterExpression} into a {@code WHERE} expression and a + * {@code Having} expression. + *

+ * {@link SplitFilterExpressionVisitor} is leveraged by the AggregationStore to construct the JPQL query. + * {@link FilterExpression} for AggregationDataStore must be split into those that apply to metric aggregations + * ({@code HAVING} clauses) and those that apply to dimensions ({@code WHERE} clauses), although only a single + * {@link FilterExpression} is passed to the datastore in each query. The split groups {@code HAVING} clauses and + * {@code WHERE} clauses separately. For example: + * 

+ * +-------------------+-----------------------------+------------------------+
+ * |    Expression     |             SQL             | WHERE-clause Promotion |
+ * +-------------------+-----------------------------+------------------------+
+ * | H1 AND W1 AND H2  | WHERE W1 HAVING (H1 AND H2) | No                     |
+ * | (H1 OR H2) AND W1 | WHERE W1 HAVING (H1 OR H2)  | No                     |
+ * | H1 OR W1          | HAVING (H1 OR W1)           | Yes                    |
+ * | (W1 AND H1) OR W2 | HAVING ((W1 AND H1) OR W2)  | Yes                    |
+ * +-------------------+-----------------------------+------------------------+
+ *
+ * Note that {@link SplitFilterExpressionVisitor} might incur more-than-expected network I/O in the case of WHERE-clause + * promotion. + *

+ * {@link SplitFilterExpressionVisitor} splits by storing {@code WHERE} and {@code HAVING} clauses in + * {@link Pair#getLeft() left} and {@link Pair#getRight() right} of a {@link Pair}, respectively. For example: + * 

+ * {@code
+ * Pair filterPair = filterExpression.accept(splitFilterExpressionVisitor);
+ *
+ * FilterExpression whereClauseFilter = filterPair.getLeft();
+ * FilterExpression havingClauseFilter = filterPair.getRight();
+ * }
+ *
+ * {@link SplitFilterExpressionVisitor} is thread-safe and can be accessed by multiple threads at the same time. + */ +@Slf4j +public class SplitFilterExpressionVisitor implements FilterExpressionVisitor { + + @Getter(value = AccessLevel.PRIVATE) + private final Table table; + + @Getter(value = AccessLevel.PRIVATE) + private final FilterExpressionNormalizationVisitor normalizationVisitor; + + /** + * Constructor. + * + * @param table Object that offers meta information about an entity field + * + * @throws NullPointerException if any one of the argument is {@code null} + */ + public SplitFilterExpressionVisitor(final Table table) { + this.table = Objects.requireNonNull(table, "table"); + this.normalizationVisitor = new FilterExpressionNormalizationVisitor(); + } + + @Override + public FilterConstraints visitPredicate(final FilterPredicate filterPredicate) { + return isHavingPredicate(filterPredicate) + ? FilterConstraints.pureHaving(filterPredicate) // this filterPredicate belongs to a HAVING clause + : FilterConstraints.pureWhere(filterPredicate); // this filterPredicate belongs to a WHERE clause + } + + @Override + public FilterConstraints visitAndExpression(final AndFilterExpression expression) { + /* + * Definition: + * C = condition + * pure-W = WHERE C + * pure-H = HAVING C + * mix-HW = WHERE C HAVING C' + * + * Given that L and R operands of an AndFilterExpression can only be one of "pure-H", "pure-W", or "mix-HW", + * then: + * + * pure-W1 AND pure-W2 = WHERE C1 AND WHERE C2 = WHERE (C1 AND C2) = pure-W + * pure-H1 AND pure-H2 = HAVING C1 AND HAVING C2 = HAVING (C1 AND C2) = pure-H + * + * pure-H1 AND pureW2 = HAVING C1 AND WHERE C2 = WHERE C2 HAVING C1 = mix-HW + * pure-W1 AND pureH2 = WHERE C1 HAVING C2 = mix-HW + * + * mix-HW1 AND pure-W2 = WHERE C1 HAVING C1' AND WHERE C2 = WHERE (C1 & C2) HAVING C1' = mix-HW + * mix-HW1 AND pure-H2 = WHERE C1 HAVING C1' AND HAVING C2 = WHERE C1 HAVING (C1' & C2) = mix-HW + * + * mix-HW1 AND mim-HW2 = WHERE C1 HAVING C1' AND WHERE C2 HAVING C2' = WHERE (C1 & C2) HAVING (C1' & C2') + * = mix-HW + */ + + FilterConstraints left = expression.getLeft().accept(this); + FilterConstraints right = expression.getRight().accept(this); + + if (left.isPureWhere() && right.isPureWhere()) { + // pure-W1 AND pure-W2 = WHERE (C1 & C2) = pure-W + return FilterConstraints.pureWhere( + new AndFilterExpression( + left.getWhereExpression(), + right.getWhereExpression() + ) + ); + } else if (left.isPureHaving() && right.isPureHaving()) { + // pure-H1 AND pure-H2 = HAVING (C1 AND C2) = pure-H + return FilterConstraints.pureHaving( + new AndFilterExpression( + left.getHavingExpression(), + right.getHavingExpression() + ) + ); + } else { + // all of the rests are mix-HW + return FilterConstraints.withWhereAndHaving( + AndFilterExpression.fromPair( + left.getWhereExpression(), + right.getWhereExpression() + ), + AndFilterExpression.fromPair( + left.getHavingExpression(), + right.getHavingExpression() + ) + ); + } + } + + @Override + public FilterConstraints visitOrExpression(final OrFilterExpression expression) { + /* + * Definition: + * C = condition + * pure-W = WHERE C + * pure-H = HAVING C + * mix-HW = WHERE C HAVING C' + * + * Given that L and R operands of an OrFilterExpression can only be one of "pure-H", "pure-W", or "mix-HW", + * then: + * + * pure-W1 OR pure-W2 = WHERE C1 OR WHERE C2 = WHERE (C1 OR C2) = pure-W + * pure-H1 OR pure-H2 = HAVING C1 OR HAVING C2 = HAVING (C1 OR C2) = pure-H + * + * pure-H1 OR pureW2 = HAVING C1 OR WHERE C2 = HAVING (C1 OR C2) = pure-H + * pure-W1 OR pureH2 = WHERE C1 OR HAVING C2 = HAVING (C1 OR C2) = pure-H + * + * mix-HW1 OR pure-W2 = (WHERE C1 HAVING C1') OR WHERE C2 = HAVING (C1 & C1' | C2) = pure-H + * mix-HW1 OR pure-H2 = (WHERE C1 HAVING C1') OR HAVING C2 = HAVING (C1 & C1' | C2) = pure-H + * + * mix-HW1 OR mim-HW2 = (WHERE C1 HAVING C1') OR (WHERE C2 HAVING C2') = HAVING ((C1 & C1') | (C2 & C2')) + * = pure-H + */ + + FilterConstraints left = expression.getLeft().accept(this); + FilterConstraints right = expression.getRight().accept(this); + + if (left.isPureWhere() && right.isPureWhere()) { + // pure-W1 OR pure-W2 = WHERE (C1 OR C2) = pure-W + return FilterConstraints.pureWhere( + OrFilterExpression.fromPair( + left.getWhereExpression(), + right.getWhereExpression() + ) + ); + } else { + // all of the rests are pure-H + return FilterConstraints.pureHaving( + OrFilterExpression.fromPair( + AndFilterExpression.fromPair( + left.getWhereExpression(), + left.getHavingExpression() + ), + AndFilterExpression.fromPair( + right.getWhereExpression(), + right.getHavingExpression() + ) + ) + ); + } + } + + @Override + public FilterConstraints visitNotExpression(NotFilterExpression expression) { + FilterExpression normalized = getNormalizationVisitor().visitNotExpression(expression); + + if (normalized instanceof AndFilterExpression) { + return visitAndExpression((AndFilterExpression) normalized); + } else if (normalized instanceof OrFilterExpression) { + return visitOrExpression((OrFilterExpression) normalized); + } else if (normalized instanceof NotFilterExpression) { + FilterConstraints negatedConstraint = visitNotExpression((NotFilterExpression) normalized); + + if (negatedConstraint.isPureWhere()) { + return FilterConstraints.pureWhere(new NotFilterExpression(negatedConstraint.getWhereExpression())); + } else { + // It is not possible to have a mixed where/having for a NotFilterExpression after normalization + // so this must be a pure HAVING + return FilterConstraints.pureHaving(new NotFilterExpression(negatedConstraint.getHavingExpression())); + } + } else { + return visitPredicate((FilterPredicate) normalized); + } + } + + /** + * Returns whether or not a {@link FilterPredicate} corresponds to a {@code HAVING} clause in JPQL query. + *

+ * A {@link FilterPredicate} corresponds to a {@code HAVING} clause iff the predicate field has + * {@link MetricAggregation} or MetricComputation annotation on it. + * + * @param filterPredicate The terminal filter expression to check for + * + * @return {@code true} if the {@link FilterPredicate} is a HAVING clause + */ + private boolean isHavingPredicate(final FilterPredicate filterPredicate) { + String fieldName = filterPredicate.getField(); + + return getTable().isMetric(fieldName); + } +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/MetaDataStore.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/MetaDataStore.java new file mode 100644 index 0000000000..eaeb3ab5fa --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/MetaDataStore.java @@ -0,0 +1,183 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.metadata; + +import com.yahoo.elide.core.EntityDictionary; +import com.yahoo.elide.core.datastore.inmemory.HashMapDataStore; +import com.yahoo.elide.core.exceptions.DuplicateMappingException; +import com.yahoo.elide.datastores.aggregation.AggregationDataStore; +import com.yahoo.elide.datastores.aggregation.annotation.MetricAggregation; +import com.yahoo.elide.datastores.aggregation.metadata.models.AnalyticView; +import com.yahoo.elide.datastores.aggregation.metadata.models.Column; +import com.yahoo.elide.datastores.aggregation.metadata.models.DataType; +import com.yahoo.elide.datastores.aggregation.metadata.models.FunctionArgument; +import com.yahoo.elide.datastores.aggregation.metadata.models.Metric; +import com.yahoo.elide.datastores.aggregation.metadata.models.MetricFunction; +import com.yahoo.elide.datastores.aggregation.metadata.models.Table; +import com.yahoo.elide.datastores.aggregation.metadata.models.TimeDimension; +import com.yahoo.elide.datastores.aggregation.metadata.models.TimeDimensionGrain; +import com.yahoo.elide.datastores.aggregation.queryengines.sql.annotation.FromSubquery; +import com.yahoo.elide.datastores.aggregation.queryengines.sql.annotation.FromTable; +import com.yahoo.elide.utils.ClassScanner; + +import org.hibernate.annotations.Subselect; + +import java.util.HashMap; +import java.util.Set; +import java.util.stream.Collectors; + +/** + * MetaDataStore is a in-memory data store that manage data models for an {@link AggregationDataStore}. + */ +public class MetaDataStore extends HashMapDataStore { + public static final Package META_DATA_PACKAGE = + Package.getPackage("com.yahoo.elide.datastores.aggregation.metadata.models"); + + private static final Class[] METADATA_STORE_ANNOTATIONS = { + FromTable.class, FromSubquery.class, Subselect.class, javax.persistence.Table.class}; + + public MetaDataStore() { + super(META_DATA_PACKAGE); + + this.dictionary = new EntityDictionary(new HashMap<>()); + + ClassScanner.getAllClasses(Table.class.getPackage().getName()).forEach(cls -> dictionary.bindEntity(cls)); + + Set> modelsToBind = ClassScanner.getAnnotatedClasses(METADATA_STORE_ANNOTATIONS); + + // bind data models in the package + modelsToBind.forEach(modelClass -> { + dictionary.bindEntity(modelClass); + }); + + // resolve meta data from the bound models + modelsToBind.forEach(modelClass -> { + addTable(isAnalyticView(modelClass) + ? new AnalyticView(modelClass, dictionary) + : new Table(modelClass, dictionary)); + }); + } + + @Override + public void populateEntityDictionary(EntityDictionary dictionary) { + ClassScanner.getAllClasses(META_DATA_PACKAGE.getName()).stream().forEach(cls -> { + dictionary.bindEntity(cls); + }); + } + + /** + * Add a table metadata object. + * + * @param table table metadata + */ + private void addTable(Table table) { + addMetaData(table); + table.getColumns().forEach(this::addColumn); + } + + /** + * Add a column metadata object. + * + * @param column column metadata + */ + private void addColumn(Column column) { + addMetaData(column); + addDataType(column.getDataType()); + + if (column instanceof TimeDimension) { + ((TimeDimension) column).getSupportedGrains().forEach(this::addTimeDimensionGrain); + } else if (column instanceof Metric) { + addMetricFunction(((Metric) column).getMetricFunction()); + } + } + + /** + * Add a metric function metadata object. + * + * @param metricFunction metric function metadata + */ + private void addMetricFunction(MetricFunction metricFunction) { + addMetaData(metricFunction); + metricFunction.getArguments().forEach(this::addFunctionArgument); + } + + /** + * Add a datatype metadata object. + * + * @param dataType datatype metadata + */ + private void addDataType(DataType dataType) { + addMetaData(dataType); + } + + /** + * Add a function argument metadata object. + * + * @param functionArgument function argument metadata + */ + private void addFunctionArgument(FunctionArgument functionArgument) { + addMetaData(functionArgument); + } + + /** + * Add a time dimension grain metadata object. + * + * @param timeDimensionGrain time dimension grain metadata + */ + private void addTimeDimensionGrain(TimeDimensionGrain timeDimensionGrain) { + addMetaData(timeDimensionGrain); + } + + /** + * Add a meta data object into this data store, check for duplication. + * + * @param object a meta data object + */ + private void addMetaData(Object object) { + Class cls = dictionary.lookupBoundClass(object.getClass()); + String id = dictionary.getId(object); + + if (dataStore.get(cls).containsKey(id)) { + if (!dataStore.get(cls).get(id).equals(object)) { + throw new DuplicateMappingException("Duplicated " + cls.getSimpleName() + " metadata " + id); + } + } else { + dataStore.get(cls).put(id, object); + } + } + + public Set getMetaData(Class cls) { + return dataStore.get(cls).values().stream().map(cls::cast).collect(Collectors.toSet()); + } + + /** + * Returns whether or not an entity field is a metric field. + *

+ * A field is a metric field iff that field is annotated by at least one of + *

    + *
  1. {@link MetricAggregation} + *
+ * + * @param dictionary entity dictionary in current Elide instance + * @param cls entity class + * @param fieldName The entity field + * + * @return {@code true} if the field is a metric field + */ + public static boolean isMetricField(EntityDictionary dictionary, Class cls, String fieldName) { + return dictionary.attributeOrRelationAnnotationExists(cls, fieldName, MetricAggregation.class); + } + + /** + * Returns whether an entity class is analytic view. + * + * @param cls entity class + * @return True if {@link FromTable} or {@link FromSubquery} is presented. + */ + private static boolean isAnalyticView(Class cls) { + return cls.isAnnotationPresent(FromTable.class) || cls.isAnnotationPresent(FromSubquery.class); + } +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/enums/Aggregation.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/enums/Aggregation.java new file mode 100644 index 0000000000..828f00b523 --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/enums/Aggregation.java @@ -0,0 +1,15 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.metadata.enums; + +/** + * Aggregation functions. + */ +public enum Aggregation { + SUM, + MIN, + MAX; +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/enums/Format.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/enums/Format.java new file mode 100644 index 0000000000..1c164e8025 --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/enums/Format.java @@ -0,0 +1,14 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ + +package com.yahoo.elide.datastores.aggregation.metadata.enums; + +/** + * Format of a value field, e.g. decimal for numbers + */ +public enum Format { + NONE +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/enums/Tag.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/enums/Tag.java new file mode 100644 index 0000000000..92065a06d3 --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/enums/Tag.java @@ -0,0 +1,13 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.metadata.enums; + +/** + * Tag attached to fields. + */ +public enum Tag { + DISPLAY +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/enums/ValueType.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/enums/ValueType.java new file mode 100644 index 0000000000..6df6643e9e --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/enums/ValueType.java @@ -0,0 +1,19 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.metadata.enums; + +/** + * Actual value type of a data type. + */ +public enum ValueType { + DATE, + NUMBER, + TEXT, + COORDINATE, + BOOLEAN, + RELATIONSHIP, + ID +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/metric/MetricFunctionInvocation.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/metric/MetricFunctionInvocation.java new file mode 100644 index 0000000000..b9dd81dcab --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/metric/MetricFunctionInvocation.java @@ -0,0 +1,69 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.metadata.metric; + +import com.yahoo.elide.datastores.aggregation.metadata.models.MetricFunction; +import com.yahoo.elide.request.Argument; + +import com.google.common.base.Functions; + +import java.util.List; +import java.util.Map; +import java.util.stream.Collectors; + +/** + * An invoked metric function instance applied on an aggregated field with provided arguments to project the result + * as the alias. + */ +public interface MetricFunctionInvocation { + /** + * Get all arguments provided for this metric function. + * + * @return request arguments + */ + List getArguments(); + + /** + * Get a name-argument map contains all arguments. + * + * @return argument map + */ + default Map getArgumentMap() { + return getArguments().stream() + .collect(Collectors.toMap(Argument::getName, Functions.identity())); + } + + /** + * Get argument for a specific name. + * + * @param argumentName argument name + * @return an argument + */ + Argument getArgument(String argumentName); + + /** + * Get invoked metric function. + * + * @return metric function + */ + MetricFunction getFunction(); + + /** + * Get full expression with provided arguments. + * + * @return function expression + */ + default String getFunctionExpression() { + return getFunction().constructExpression(getArgumentMap()); + } + + /** + * Get alias of this invocation. + * + * @return alias + */ + String getAlias(); +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/AnalyticView.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/AnalyticView.java new file mode 100644 index 0000000000..65b03048bb --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/AnalyticView.java @@ -0,0 +1,48 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.metadata.models; + +import com.yahoo.elide.annotation.Include; +import com.yahoo.elide.core.EntityDictionary; + +import lombok.Data; +import lombok.EqualsAndHashCode; +import lombok.ToString; + +import java.util.Set; +import java.util.stream.Collectors; +import javax.persistence.OneToMany; + +/** + * AnalyticViews are logical tables that support aggregation functionality, but don't support join or relationship. + */ +@EqualsAndHashCode(callSuper = true) +@Include(rootLevel = true, type = "analyticView") +@Data +public class AnalyticView extends Table { + + @OneToMany + @ToString.Exclude + private Set metrics; + + @OneToMany + @ToString.Exclude + private Set dimensions; + + public AnalyticView(Class cls, EntityDictionary dictionary) { + super(cls, dictionary); + + metrics = getColumns().stream() + .filter(col -> col instanceof Metric) + .map(Metric.class::cast) + .collect(Collectors.toSet()); + + dimensions = getColumns().stream() + .filter(col -> !(col instanceof Metric)) + .map(Dimension.class::cast) + .collect(Collectors.toSet()); + } +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/Column.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/Column.java new file mode 100644 index 0000000000..8bc9c02baf --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/Column.java @@ -0,0 +1,87 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.metadata.models; + +import com.yahoo.elide.annotation.Include; +import com.yahoo.elide.core.EntityDictionary; +import com.yahoo.elide.datastores.aggregation.annotation.Meta; +import com.yahoo.elide.datastores.aggregation.metadata.enums.Tag; +import com.yahoo.elide.datastores.aggregation.metadata.enums.ValueType; + +import lombok.Data; +import lombok.ToString; + +import java.util.Date; +import java.util.HashSet; +import java.util.Locale; +import java.util.Set; +import javax.persistence.Id; +import javax.persistence.ManyToOne; + +/** + * Column is the super class of a field in a table, it can be either dimension or metric. + */ +@Include(type = "column") +@Data +@ToString +public abstract class Column { + @Id + private String id; + + private String name; + + private String longName; + + private String tableName; + + private String description; + + private String category; + + @ManyToOne + private DataType dataType; + + @ToString.Exclude + private Set columnTags; + + protected Column(Class tableClass, String fieldName, EntityDictionary dictionary) { + this.tableName = dictionary.getJsonAliasFor(tableClass); + this.id = tableName + "." + fieldName; + this.name = fieldName; + this.columnTags = new HashSet<>(); + + Meta meta = dictionary.getAttributeOrRelationAnnotation(tableClass, Meta.class, fieldName); + if (meta != null) { + this.longName = meta.longName(); + this.description = meta.description(); + } + + dataType = getDataType(tableClass, fieldName, dictionary); + if (dataType == null) { + throw new IllegalArgumentException("Unknown data type for " + this.id); + } + } + + public static DataType getDataType(Class tableClass, String fieldName, EntityDictionary dictionary) { + String tableName = dictionary.getJsonAliasFor(tableClass); + DataType dataType; + if (dictionary.isRelation(tableClass, fieldName)) { + Class relationshipClass = dictionary.getParameterizedType(tableClass, fieldName); + dataType = new RelationshipType(dictionary.getJsonAliasFor(relationshipClass)); + } else { + Class fieldClass = dictionary.getType(tableClass, fieldName); + + if (fieldName.equals(dictionary.getIdFieldName(tableClass))) { + dataType = new DataType(tableName + "." + fieldName, ValueType.ID); + } else if (Date.class.isAssignableFrom(fieldClass)) { + dataType = new DataType(fieldClass.getSimpleName().toLowerCase(Locale.ENGLISH), ValueType.DATE); + } else { + dataType = DataType.getScalarType(fieldClass); + } + } + return dataType; + } +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/DataType.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/DataType.java new file mode 100644 index 0000000000..de8a0d2b81 --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/DataType.java @@ -0,0 +1,58 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.metadata.models; + +import static com.yahoo.elide.datastores.aggregation.metadata.enums.ValueType.BOOLEAN; +import static com.yahoo.elide.datastores.aggregation.metadata.enums.ValueType.NUMBER; +import static com.yahoo.elide.datastores.aggregation.metadata.enums.ValueType.TEXT; + +import com.yahoo.elide.annotation.Include; +import com.yahoo.elide.datastores.aggregation.metadata.enums.ValueType; + +import lombok.AllArgsConstructor; +import lombok.Data; +import lombok.ToString; + +import java.math.BigDecimal; +import java.util.HashMap; +import java.util.Map; +import javax.persistence.Id; + +/** + * Data type of a column. + */ +@Include(rootLevel = true, type = "dataType") +@Data +@AllArgsConstructor +@ToString +public class DataType { + private static final Map, DataType> SCALAR_TYPES = new HashMap, DataType>() {{ + put(short.class, new DataType("p_short", NUMBER)); + put(Short.class, new DataType("short", NUMBER)); + put(int.class, new DataType("p_int", NUMBER)); + put(Integer.class, new DataType("int", NUMBER)); + put(long.class, new DataType("p_bigint", NUMBER)); + put(Long.class, new DataType("bigint", NUMBER)); + put(BigDecimal.class, new DataType("bigDecimal", NUMBER)); + put(float.class, new DataType("p_float", NUMBER)); + put(Float.class, new DataType("float", NUMBER)); + put(double.class, new DataType("p_double", NUMBER)); + put(Double.class, new DataType("double", NUMBER)); + put(boolean.class, new DataType("p_boolean", BOOLEAN)); + put(Boolean.class, new DataType("boolean", BOOLEAN)); + put(char.class, new DataType("p_char", TEXT)); + put(String.class, new DataType("string", TEXT)); + }}; + + @Id + private String name; + + private ValueType valueType; + + public static DataType getScalarType(Class valueClass) { + return SCALAR_TYPES.get(valueClass); + } +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/Dimension.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/Dimension.java new file mode 100644 index 0000000000..3ed5f7f56a --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/Dimension.java @@ -0,0 +1,24 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.metadata.models; + +import com.yahoo.elide.annotation.Include; +import com.yahoo.elide.core.EntityDictionary; + +import lombok.Data; +import lombok.EqualsAndHashCode; + +/** + * Regular field in tables, can be grouped by if the table is an AnalyticView. + */ +@EqualsAndHashCode(callSuper = true) +@Include(type = "dimension") +@Data +public class Dimension extends Column { + public Dimension(Class tableClass, String fieldName, EntityDictionary dictionary) { + super(tableClass, fieldName, dictionary); + } +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/FunctionArgument.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/FunctionArgument.java new file mode 100644 index 0000000000..d7b1de386b --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/FunctionArgument.java @@ -0,0 +1,39 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.metadata.models; + +import com.yahoo.elide.annotation.Include; + +import lombok.Data; +import lombok.ToString; + +import javax.persistence.Id; +import javax.persistence.ManyToOne; + +/** + * Arguments that can be provided into a metric function. + */ +@Include(type = "functionArgument") +@Data +@ToString +public class FunctionArgument { + @Id + private String id; + + private String name; + + private String description; + + @ManyToOne + private DataType dataType; + + public FunctionArgument(String functionName, FunctionArgument argument) { + this.id = functionName + "." + argument.getName(); + this.name = argument.getName(); + this.description = argument.getDescription(); + this.dataType = argument.getDataType(); + } +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/Metric.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/Metric.java new file mode 100644 index 0000000000..8dcfaf3d16 --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/Metric.java @@ -0,0 +1,50 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.metadata.models; + +import com.yahoo.elide.annotation.Include; +import com.yahoo.elide.core.EntityDictionary; +import com.yahoo.elide.datastores.aggregation.annotation.MetricAggregation; +import com.yahoo.elide.datastores.aggregation.metadata.enums.Format; + +import lombok.Data; +import lombok.EqualsAndHashCode; +import lombok.ToString; + +import javax.persistence.ManyToOne; + +/** + * Special column for AnalyticView which supports aggregation. + */ +@EqualsAndHashCode(callSuper = true) +@Include(type = "metric") +@Data +public class Metric extends Column { + private Format defaultFormat; + + @ManyToOne + @ToString.Exclude + private MetricFunction metricFunction; + + public Metric(Class tableClass, String fieldName, EntityDictionary dictionary) { + super(tableClass, fieldName, dictionary); + + MetricAggregation metric = dictionary.getAttributeOrRelationAnnotation( + tableClass, + MetricAggregation.class, + fieldName); + + try { + this.metricFunction = metric.function().newInstance(); + metricFunction.setName(getId() + "[" + metricFunction.getName() + "]"); + metricFunction.setExpression(String.format( + metricFunction.getExpression(), + dictionary.getAnnotatedColumnName(tableClass, fieldName))); + } catch (InstantiationException | IllegalAccessException e) { + throw new IllegalArgumentException("Can't initialize function for metric " + getId()); + } + } +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/MetricFunction.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/MetricFunction.java new file mode 100644 index 0000000000..6c3b06ed6b --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/MetricFunction.java @@ -0,0 +1,114 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.metadata.models; + +import com.yahoo.elide.annotation.Include; +import com.yahoo.elide.core.exceptions.InvalidPredicateException; +import com.yahoo.elide.datastores.aggregation.metadata.metric.MetricFunctionInvocation; +import com.yahoo.elide.request.Argument; + +import lombok.AllArgsConstructor; +import lombok.Data; +import lombok.ToString; + +import java.util.ArrayList; +import java.util.List; +import java.util.Map; +import java.util.Set; +import java.util.function.Function; +import java.util.stream.Collectors; +import javax.persistence.Id; +import javax.persistence.OneToMany; + +/** + * Functions used to compute metrics. + */ +@Include(type = "metricFunction") +@Data +@ToString +@AllArgsConstructor +public class MetricFunction { + @Id + private String name; + + private String longName; + + private String description; + + private String expression; + + @OneToMany + private Set arguments; + + protected MetricFunctionInvocation invoke(Map arguments, String alias) { + final MetricFunction function = this; + return new MetricFunctionInvocation() { + @Override + public List getArguments() { + return new ArrayList<>(arguments.values()); + } + + @Override + public Argument getArgument(String argumentName) { + return arguments.get(argumentName); + } + + @Override + public MetricFunction getFunction() { + return function; + } + + @Override + public String getAlias() { + return alias; + } + }; + } + + /** + * Construct full metric expression using arguments. + * + * @param arguments provided arguments + * @return FUNCTION(field1, field2, ..., arg1, arg2, ...) + */ + public String constructExpression(Map arguments) { + return getExpression(); + } + + /** + * Get all required argument names for this metric function. + * + * @return all argument names + */ + private Set getArgumentNames() { + return getArguments().stream().map(FunctionArgument::getName).collect(Collectors.toSet()); + } + + /** + * Invoke this metric function with arguments, an aggregated field and projection alias. + * + * @param arguments arguments provided in the request + * @param alias result alias + * @return an invoked metric function + */ + public final MetricFunctionInvocation invoke(Set arguments, String alias) { + Set requiredArguments = getArgumentNames(); + Set providedArguments = arguments.stream() + .map(Argument::getName) + .collect(Collectors.toSet()); + + if (!requiredArguments.equals(providedArguments)) { + throw new InvalidPredicateException( + "Provided arguments doesn't match requirement for function " + getName() + "."); + } + + // map arguments to their actual name + Map resolvedArguments = arguments.stream() + .collect(Collectors.toMap(Argument::getName, Function.identity())); + + return invoke(resolvedArguments, alias); + } +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/RelationshipType.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/RelationshipType.java new file mode 100644 index 0000000000..c0e8f49587 --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/RelationshipType.java @@ -0,0 +1,22 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.metadata.models; + +import com.yahoo.elide.datastores.aggregation.metadata.enums.ValueType; + +import lombok.Data; +import lombok.EqualsAndHashCode; + +/** + * Special data type that represents a relationship between tables. + */ +@Data +@EqualsAndHashCode(callSuper = true) +public class RelationshipType extends DataType { + public RelationshipType(String name) { + super(name, ValueType.RELATIONSHIP); + } +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/Table.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/Table.java new file mode 100644 index 0000000000..b31af3e474 --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/Table.java @@ -0,0 +1,148 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.metadata.models; + +import static com.yahoo.elide.datastores.aggregation.metadata.MetaDataStore.isMetricField; + +import com.yahoo.elide.annotation.Include; +import com.yahoo.elide.core.EntityDictionary; +import com.yahoo.elide.datastores.aggregation.annotation.Cardinality; +import com.yahoo.elide.datastores.aggregation.annotation.CardinalitySize; +import com.yahoo.elide.datastores.aggregation.annotation.Meta; +import com.yahoo.elide.datastores.aggregation.annotation.Temporal; + +import lombok.Data; +import lombok.ToString; + +import java.util.Set; +import java.util.stream.Collectors; +import javax.persistence.Id; +import javax.persistence.OneToMany; +import javax.persistence.Transient; + +/** + * Super class of all logical or physical tables. + */ +@Include(rootLevel = true, type = "table") +@Data +@ToString +public class Table { + @Transient + private Class cls; + + @Id + private String name; + + private String longName; + + private String description; + + private String category; + + private CardinalitySize cardinality; + + @OneToMany + @ToString.Exclude + private Set columns; + + public Table(Class cls, EntityDictionary dictionary) { + if (!dictionary.getBindings().contains(cls)) { + throw new IllegalArgumentException( + String.format("Table class {%s} is not defined in dictionary.", cls)); + } + + this.cls = cls; + this.name = dictionary.getJsonAliasFor(cls); + + this.columns = resolveColumns(cls, dictionary); + + Meta meta = cls.getAnnotation(Meta.class); + if (meta != null) { + this.longName = meta.longName(); + this.description = meta.description(); + } + + Cardinality cardinality = dictionary.getAnnotation(cls, Cardinality.class); + if (cardinality != null) { + this.cardinality = cardinality.size(); + } + } + + /** + * Add all columns of this table. + * + * @param cls table class + * @param dictionary dictionary contains the table class + * @return all resolved column metadata + */ + private static Set resolveColumns(Class cls, EntityDictionary dictionary) { + Set fields = dictionary.getAllFields(cls).stream() + .filter((field) -> Column.getDataType(cls, field, dictionary) != null) + .map(field -> { + if (isMetricField(dictionary, cls, field)) { + return new Metric(cls, field, dictionary); + } else if (dictionary.attributeOrRelationAnnotationExists(cls, field, Temporal.class)) { + return new TimeDimension(cls, field, dictionary); + } else { + return new Dimension(cls, field, dictionary); + } + }) + .collect(Collectors.toSet()); + + // add id field if exists + if (dictionary.getIdFieldName(cls) != null) { + fields.add(new Dimension(cls, dictionary.getIdFieldName(cls), dictionary)); + } + + return fields; + } + + /** + * Get all columns of a specific class, can be {@link Metric}, {@link TimeDimension} or {@link Dimension}. + * + * @param cls metadata class + * @param metadata class + * @return columns as requested type if found + */ + public Set getColumns(Class cls) { + return columns.stream() + .filter(col -> cls.isAssignableFrom(col.getClass())) + .map(cls::cast) + .collect(Collectors.toSet()); + } + + /** + * Get a field column as a specific class, can be {@link Metric}, {@link TimeDimension} or {@link Dimension}. + * + * @param cls metadata class + * @param fieldName logical column name + * @param metadata class + * @return column as requested type if found + */ + private T getColumn(Class cls, String fieldName) { + return columns.stream() + .filter(col -> cls.isAssignableFrom(col.getClass()) && (col.getName().equals(fieldName))) + .map(cls::cast) + .findFirst() + .orElse(null); + } + + public Metric getMetric(String fieldName) { + return getColumn(Metric.class, fieldName); + } + + public Dimension getDimension(String fieldName) { + return getColumn(Dimension.class, fieldName); + } + + public TimeDimension getTimeDimension(String fieldName) { + return getColumn(TimeDimension.class, fieldName); + } + + public boolean isMetric(String fieldName) { + return getMetric(fieldName) != null; + } +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/TimeDimension.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/TimeDimension.java new file mode 100644 index 0000000000..22b06e3fd8 --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/TimeDimension.java @@ -0,0 +1,44 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.metadata.models; + +import com.yahoo.elide.annotation.Include; +import com.yahoo.elide.core.EntityDictionary; +import com.yahoo.elide.datastores.aggregation.annotation.Temporal; + +import lombok.Data; +import lombok.EqualsAndHashCode; + +import java.util.Arrays; +import java.util.LinkedHashSet; +import java.util.Set; +import java.util.TimeZone; +import java.util.stream.Collectors; +import javax.persistence.ManyToMany; + +/** + * TimeDimension is a dimension that represents time value. + * This type of dimension can be used to support more specific aggregation logic e.g. DAILY/MONTHLY aggregation + */ +@EqualsAndHashCode(callSuper = true) +@Include(type = "timeDimension") +@Data +public class TimeDimension extends Dimension { + @ManyToMany + Set supportedGrains; + + private TimeZone timezone; + + public TimeDimension(Class tableClass, String fieldName, EntityDictionary dictionary) { + super(tableClass, fieldName, dictionary); + + Temporal temporal = dictionary.getAttributeOrRelationAnnotation(tableClass, Temporal.class, fieldName); + + this.supportedGrains = Arrays.stream(temporal.grains()) + .map(grain -> new TimeDimensionGrain(getId(), grain)) + .collect(Collectors.toCollection(LinkedHashSet::new)); + } +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/TimeDimensionGrain.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/TimeDimensionGrain.java new file mode 100644 index 0000000000..7ff87e8380 --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/TimeDimensionGrain.java @@ -0,0 +1,35 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.metadata.models; + +import com.yahoo.elide.annotation.Include; +import com.yahoo.elide.datastores.aggregation.annotation.TimeGrainDefinition; +import com.yahoo.elide.datastores.aggregation.time.TimeGrain; + +import lombok.Data; + +import java.util.Locale; +import javax.persistence.Id; + +/** + * Defines how to extract a time dimension for a specific grain from a table. + */ +@Include(type = "timeDimensionGrain") +@Data +public class TimeDimensionGrain { + @Id + private String id; + + private TimeGrain grain; + + private String expression; + + public TimeDimensionGrain(String fieldName, TimeGrainDefinition definition) { + this.id = fieldName + "." + definition.grain().name().toLowerCase(Locale.ENGLISH); + this.grain = definition.grain(); + this.expression = definition.expression(); + } +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/query/ColumnProjection.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/query/ColumnProjection.java new file mode 100644 index 0000000000..2228751f72 --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/query/ColumnProjection.java @@ -0,0 +1,91 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.query; + +import com.yahoo.elide.core.exceptions.InvalidValueException; +import com.yahoo.elide.datastores.aggregation.metadata.models.Column; +import com.yahoo.elide.datastores.aggregation.metadata.models.Dimension; +import com.yahoo.elide.datastores.aggregation.metadata.models.TimeDimension; +import com.yahoo.elide.datastores.aggregation.time.TimeGrain; + +import java.io.Serializable; +import java.util.TimeZone; + +/** + * Represents a projected column as an alias in a query. + */ +public interface ColumnProjection extends Serializable { + /** + * Get the projected column. + * + * @return column + */ + Column getColumn(); + + /** + * Get the projection alias. + * + * @return alias + */ + String getAlias(); + + /** + * Project a dimension as alias. + * + * @param dimension dimension column + * @param alias alias + * @return a projection represents that "dimension AS alias" + */ + static ColumnProjection toProjection(Dimension dimension, String alias) { + return new ColumnProjection() { + @Override + public Dimension getColumn() { + return dimension; + } + + @Override + public String getAlias() { + return alias; + } + }; + } + + /** + * Project a time dimension as alias with specific time grain. + * + * @param dimension time dimension column + * @param grain projected time grain + * @param alias alias + * @return a projection represents that "grain(dimension) AS alias" + */ + static TimeDimensionProjection toProjection(TimeDimension dimension, TimeGrain grain, String alias) { + // TODO: get time zone from the request + if (dimension.getSupportedGrains().stream().anyMatch(g -> g.getGrain().equals(grain))) { + return new TimeDimensionProjection() { + @Override + public TimeGrain getGrain() { + return grain; + } + + @Override + public TimeZone getTimeZone() { + return null; + } + + @Override + public TimeDimension getTimeDimension() { + return dimension; + } + + @Override + public String getAlias() { + return alias; + } + }; + } + throw new InvalidValueException(dimension.getId() + " doesn't support grain " + grain); + } +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/query/Query.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/query/Query.java new file mode 100644 index 0000000000..9d6687ca79 --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/query/Query.java @@ -0,0 +1,57 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.query; + +import com.yahoo.elide.core.RequestScope; +import com.yahoo.elide.core.filter.expression.FilterExpression; +import com.yahoo.elide.core.pagination.Pagination; +import com.yahoo.elide.core.sort.Sorting; +import com.yahoo.elide.datastores.aggregation.QueryEngine; +import com.yahoo.elide.datastores.aggregation.metadata.metric.MetricFunctionInvocation; +import com.yahoo.elide.datastores.aggregation.metadata.models.AnalyticView; + +import lombok.Builder; +import lombok.Data; +import lombok.Singular; + +import java.util.LinkedHashSet; +import java.util.List; +import java.util.Set; +import java.util.stream.Collectors; +import java.util.stream.Stream; + +/** + * A {@link Query} is an object representing a query executed by {@link QueryEngine}. + */ +@Data +@Builder +public class Query { + private final AnalyticView analyticView; + + @Singular + private final List metrics; + + @Singular + private final Set groupByDimensions; + + @Singular + private final Set timeDimensions; + + private final FilterExpression whereFilter; + private final FilterExpression havingFilter; + private final Sorting sorting; + private final Pagination pagination; + private final RequestScope scope; + + /** + * Returns all the dimensions regardless of type. + * @return All the dimensions. + */ + public Set getDimensions() { + return Stream.concat(getGroupByDimensions().stream(), getTimeDimensions().stream()) + .collect(Collectors.toCollection(LinkedHashSet::new)); + } +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/query/TimeDimensionProjection.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/query/TimeDimensionProjection.java new file mode 100644 index 0000000000..bab8385207 --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/query/TimeDimensionProjection.java @@ -0,0 +1,86 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ + +package com.yahoo.elide.datastores.aggregation.query; + +import com.yahoo.elide.core.exceptions.InvalidValueException; +import com.yahoo.elide.datastores.aggregation.metadata.models.Column; +import com.yahoo.elide.datastores.aggregation.metadata.models.TimeDimension; +import com.yahoo.elide.datastores.aggregation.time.TimeGrain; + +import java.util.TimeZone; + +/** + * Represents a requested time dimension in a query. + */ +public interface TimeDimensionProjection extends ColumnProjection { + /** + * Get the projected time dimension. + * + * @return time dimension + */ + TimeDimension getTimeDimension(); + + /** + * The time dimension is the projected column. + * + * @return project column + */ + @Override + default Column getColumn() { + return getTimeDimension(); + } + + /** + * Get the requested time grain. + * + * @return time grain + */ + TimeGrain getGrain(); + + /** + * Get the requested time zone. + * + * @return time zone + */ + TimeZone getTimeZone(); + + /** + * Convert this projection to a new time grain. + * + * @param newGrain new time grain + * @return a new projection + */ + default TimeDimensionProjection toTimeGrain(TimeGrain newGrain) { + if (getTimeDimension().getSupportedGrains().stream() + .noneMatch(supportedGrain -> supportedGrain.getGrain().equals(newGrain))) { + throw new InvalidValueException(getTimeDimension().getId() + " doesn't support grain " + newGrain); + } + + TimeDimensionProjection projection = this; + return new TimeDimensionProjection() { + @Override + public TimeDimension getTimeDimension() { + return projection.getTimeDimension(); + } + + @Override + public TimeGrain getGrain() { + return newGrain; + } + + @Override + public TimeZone getTimeZone() { + return projection.getTimeZone(); + } + + @Override + public String getAlias() { + return projection.getAlias(); + } + }; + } +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/AbstractEntityHydrator.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/AbstractEntityHydrator.java new file mode 100644 index 0000000000..b8e2151a1c --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/AbstractEntityHydrator.java @@ -0,0 +1,203 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.queryengines; + +import com.yahoo.elide.core.EntityDictionary; +import com.yahoo.elide.datastores.aggregation.QueryEngine; +import com.yahoo.elide.datastores.aggregation.metadata.metric.MetricFunctionInvocation; +import com.yahoo.elide.datastores.aggregation.metadata.models.Dimension; +import com.yahoo.elide.datastores.aggregation.metadata.models.RelationshipType; +import com.yahoo.elide.datastores.aggregation.query.ColumnProjection; +import com.yahoo.elide.datastores.aggregation.query.Query; + +import com.google.common.base.Preconditions; +import org.apache.commons.lang3.mutable.MutableInt; + +import lombok.AccessLevel; +import lombok.Getter; + +import java.util.ArrayList; +import java.util.HashMap; +import java.util.List; +import java.util.Map; +import java.util.stream.Collectors; + +/** + * {@link AbstractEntityHydrator} hydrates the entity loaded by {@link QueryEngine#executeQuery(Query)}. + *

+ * {@link AbstractEntityHydrator} is not thread-safe and should be accessed by only 1 thread in this application, + * because it uses {@link StitchList}. See {@link StitchList} for more details. + */ +public abstract class AbstractEntityHydrator { + + @Getter(AccessLevel.PROTECTED) + private final EntityDictionary entityDictionary; + + @Getter(AccessLevel.PRIVATE) + private final StitchList stitchList; + + @Getter(AccessLevel.PROTECTED) + private final List> results = new ArrayList<>(); + + @Getter(AccessLevel.PRIVATE) + private final Query query; + + /** + * Constructor. + * + * @param results The loaded objects from {@link QueryEngine#executeQuery(Query)} + * @param query The query passed to {@link QueryEngine#executeQuery(Query)} to load the objects + * @param entityDictionary An object that sets entity instance values and provides entity metadata info + */ + public AbstractEntityHydrator(List results, Query query, EntityDictionary entityDictionary) { + this.stitchList = new StitchList(entityDictionary); + this.query = query; + this.entityDictionary = entityDictionary; + + //Get all the projections from the client query. + List projections = this.query.getMetrics().stream() + .map(MetricFunctionInvocation::getAlias) + .collect(Collectors.toList()); + + projections.addAll(this.query.getDimensions().stream() + .map(ColumnProjection::getAlias) + .collect(Collectors.toList())); + + results.forEach(result -> { + Map row = new HashMap<>(); + + Object[] resultValues = result instanceof Object[] ? (Object[]) result : new Object[] { result }; + + Preconditions.checkArgument(projections.size() == resultValues.length); + + for (int idx = 0; idx < resultValues.length; idx++) { + Object value = resultValues[idx]; + String fieldName = projections.get(idx); + row.put(fieldName, value); + } + + this.results.add(row); + }); + } + + /** + * Loads a map of relationship object ID to relationship object instance. + *

+ * Note the relationship cannot be toMany. This method will be invoked for every relationship field of the + * requested entity. Its implementation should return the result of the following query + *

+ * Given a relationship with type {@code relationshipType} in an entity, loads all relationship + * objects whose foreign keys are one of the specified list, {@code joinFieldIds}. + *

+ * For example, when the relationship is loaded from SQL and we have the following example identity: + * 

+     * public class PlayerStats {
+     *     private String id;
+     *     private Country country;
+     *
+     *     @OneToOne
+     *     @JoinColumn(name = "country_id")
+     *     public Country getCountry() {
+     *         return country;
+     *     }
+     * }
+     *
+ * In this case {@code relationshipType = Country.class}. If {@code country} is + * requested in {@code PlayerStats} query and 3 stats, for example, are found in database whose country ID's are + * {@code joinFieldIds = [840, 344, 840]}, then this method should effectively run the following query (JPQL as + * example) + * 
+     * {@code
+     *     SELECT e FROM country_table e WHERE country_id IN (840, 344);
+     * }
+     *
+ * and returns the map of [840: Country(id:840), 344: Country(id:344)] + * + * @param relationshipType The type of relationship + * @param joinFieldIds The specified list of join ID's against the relationship + * + * @return a list of hydrating values + */ + protected abstract Map getRelationshipValues( + Class relationshipType, + List joinFieldIds + ); + + public Iterable hydrate() { + //Coerce the results into entity objects. + MutableInt counter = new MutableInt(0); + + List queryResults = getResults().stream() + .map((result) -> coerceObjectToEntity(result, counter)) + .collect(Collectors.toList()); + + if (getStitchList().shouldStitch()) { + // relationship is requested, stitch relationship then + populateObjectLookupTable(); + getStitchList().stitch(); + } + + return queryResults; + } + + /** + * Coerces results from a {@link Query} into an Object. + * + * @param result a fieldName-value map + * @param counter Monotonically increasing number to generate IDs. + * @return A hydrated entity object. + */ + protected Object coerceObjectToEntity(Map result, MutableInt counter) { + Class entityClass = query.getAnalyticView().getCls(); + + //Construct the object. + Object entityInstance; + try { + entityInstance = entityClass.newInstance(); + } catch (InstantiationException | IllegalAccessException e) { + throw new IllegalStateException(e); + } + + result.forEach((fieldName, value) -> { + Dimension dim = query.getAnalyticView().getDimension(fieldName); + + if (dim != null && dim.getDataType() instanceof RelationshipType) { + getStitchList().todo(entityInstance, fieldName, value); // We don't hydrate relationships here. + } else { + getEntityDictionary().setValue(entityInstance, fieldName, value); + } + }); + + //Set the ID (it must be coerced from an integer) + getEntityDictionary().setValue( + entityInstance, + getEntityDictionary().getIdFieldName(entityClass), + counter.getAndIncrement() + ); + + return entityInstance; + } + + /** + * Foe each requested relationship, run a single query to load all relationship objects whose ID's are involved in + * the request. + */ + private void populateObjectLookupTable() { + // mapping: relationship field name -> join ID's + Map> hydrationIdsByRelationship = getStitchList().getHydrationMapping(); + + // hydrate each relationship + for (Map.Entry> entry : hydrationIdsByRelationship.entrySet()) { + String joinField = entry.getKey(); + List joinFieldIds = entry.getValue(); + Class relationshipType = getEntityDictionary().getParameterizedType( + getQuery().getAnalyticView().getCls(), + joinField); + + getStitchList().populateLookup(relationshipType, getRelationshipValues(relationshipType, joinFieldIds)); + } + } +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/StitchList.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/StitchList.java new file mode 100644 index 0000000000..7dd9a95b1a --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/StitchList.java @@ -0,0 +1,162 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.queryengines; + +import com.yahoo.elide.core.EntityDictionary; +import com.yahoo.elide.utils.coerce.CoerceUtil; +import lombok.AccessLevel; +import lombok.Data; +import lombok.Getter; + +import java.util.ArrayList; +import java.util.Collections; +import java.util.HashMap; +import java.util.LinkedList; +import java.util.List; +import java.util.Map; +import java.util.stream.Collectors; + +/** + * An auxiliary class for {@link AbstractEntityHydrator} and is responsible for setting relationship values of an entity + * instance. + *

+ * {@link StitchList} should not be subclassed. + */ +public final class StitchList { + /** + * Maps an relationship entity class to a map of object ID to object instance. + *

+ * For example, [Country.class: [340: Country(id:340), 100: Country(id:100)]] + */ + @Getter(AccessLevel.PRIVATE) + private final Map, Map> objectLookups; + + /** + * List of relationships to hydrate. + */ + @Getter(AccessLevel.PRIVATE) + private final List todoList; + + @Getter(AccessLevel.PRIVATE) + private final EntityDictionary entityDictionary; + + /** + * A representation of an TODO item in a {@link StitchList}. + */ + @Data + public static class Todo { + private final Object entityInstance; + private final String relationshipName; + private final Object foreignKey; + } + + /** + * Constructor. + * + * @param entityDictionary An object that sets entity instance values and provides entity metadata info + */ + public StitchList(EntityDictionary entityDictionary) { + this.objectLookups = new HashMap<>(); + this.todoList = new ArrayList<>(); + this.entityDictionary = entityDictionary; + } + + /** + * Returns whether or not the entity instances in this {@link StitchList} have relationships that are unset. + * + * @return {@code true} if the entity instances in this {@link StitchList} should be further hydrated because they + * have one or more relationship fields. + */ + public boolean shouldStitch() { + return !getTodoList().isEmpty(); + } + + /** + * Enqueues an entity instance which will be further hydrated on one of its relationship fields later. + * + * @param entityInstance The entity instance to be hydrated + * @param fieldName The relationship field to hydrate in the entity instance + * @param value The foreign key between the entity instance and the field entity. + */ + public void todo(Object entityInstance, String fieldName, Object value) { + Object coercedValue = CoerceUtil.coerce(value, getEntityDictionary() + .getIdType(getEntityDictionary().getParameterizedType(entityInstance, fieldName))); + getTodoList().add(new Todo(entityInstance, fieldName, coercedValue)); + } + + /** + * Sets all the relationship values of an requested entity. + *

+ * Values associated with the existing key will be overwritten. + * + * @param relationshipType The type of the relationship to set + * @param idToInstance A map from relationship ID to the actual relationship instance with that ID + */ + public void populateLookup(Class relationshipType, Map idToInstance) { + if (getObjectLookups().containsKey(relationshipType)) { + getObjectLookups().get(relationshipType).putAll(idToInstance); + } else { + getObjectLookups().put(relationshipType, idToInstance); + } + } + + /** + * Stitch all entity instances currently in this {@link StitchList} by setting their relationship fields whose + * values are determined by relationship ID's. + */ + public void stitch() { + for (Todo todo : getTodoList()) { + Object entityInstance = todo.getEntityInstance(); + String relationshipName = todo.getRelationshipName(); + Object foreignKey = todo.getForeignKey(); + + Class relationshipType = getEntityDictionary().getParameterizedType(entityInstance, relationshipName); + Object relationshipValue = getObjectLookups().get(relationshipType).get(foreignKey); + + getEntityDictionary().setValue(entityInstance, relationshipName, relationshipValue); + } + } + + /** + * Returns a mapping from relationship name to an immutable list of foreign key ID objects. + *

+ * For example, given the following {@code todoList}: + * 

+     * {@code
+     *     [PlayerStats, country, 344]
+     *     [PlayerStats, country, 840]
+     *     [PlayerStats, country, 344]
+     *     [PlayerStats, player, 1]
+     *     [PlayerStats, player, 1]
+     *     [PlayerStats, player, 1]
+     * }
+     *
+ * this method returns a map of the following: + * 
+     *     [
+     *         "country": [344, 840]
+     *         "player": [1]
+     *     ]
+     *
+ * + * @return a mapping from relationship name to an ordered list of relationship join ID's + */ + public Map> getHydrationMapping() { + return getTodoList().stream() + .collect( + Collectors.groupingBy( + Todo::getRelationshipName, + Collectors.mapping( + Todo::getForeignKey, + Collectors.collectingAndThen( + Collectors.toCollection(LinkedList::new), + Collections::unmodifiableList + ) + ) + ) + ); + } +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/SQLEntityHydrator.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/SQLEntityHydrator.java new file mode 100644 index 0000000000..ac93a6d920 --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/SQLEntityHydrator.java @@ -0,0 +1,83 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.queryengines.sql; + +import com.yahoo.elide.core.EntityDictionary; +import com.yahoo.elide.datastores.aggregation.query.Query; +import com.yahoo.elide.datastores.aggregation.queryengines.AbstractEntityHydrator; +import com.yahoo.elide.utils.coerce.CoerceUtil; +import lombok.AccessLevel; +import lombok.Getter; + +import java.util.AbstractMap; +import java.util.Collections; +import java.util.LinkedList; +import java.util.List; +import java.util.Map; +import java.util.stream.Collectors; + +import javax.persistence.EntityManager; + +/** + * {@link SQLEntityHydrator} hydrates the entity loaded by {@link SQLQueryEngine#executeQuery(Query)}. + */ +public class SQLEntityHydrator extends AbstractEntityHydrator { + + @Getter(AccessLevel.PRIVATE) + private final EntityManager entityManager; + + /** + * Constructor. + * + * @param results The loaded objects from {@link SQLQueryEngine#executeQuery(Query)} + * @param query The query passed to {@link SQLQueryEngine#executeQuery(Query)} to load the objects + * @param entityDictionary An object that sets entity instance values and provides entity metadata info + * @param entityManager An service that issues JPQL queries to load relationship objects + */ + public SQLEntityHydrator( + List results, + Query query, + EntityDictionary entityDictionary, + EntityManager entityManager + ) { + super(results, query, entityDictionary); + this.entityManager = entityManager; + } + + @Override + protected Map getRelationshipValues( + Class relationshipType, + List joinFieldIds + ) { + if (joinFieldIds.isEmpty()) { + return Collections.emptyMap(); + } + + List uniqueIds = joinFieldIds.stream() + .distinct() + .collect(Collectors.toCollection(LinkedList::new)); + + List loaded = getEntityManager() + .createQuery( + String.format( + "SELECT e FROM %s e WHERE %s IN (:idList)", + relationshipType.getCanonicalName(), + getEntityDictionary().getIdFieldName(relationshipType) + ) + ) + .setParameter("idList", uniqueIds) + .getResultList(); + + return loaded.stream() + .map(obj -> new AbstractMap.SimpleImmutableEntry<>( + CoerceUtil.coerce( + (Object) getEntityDictionary().getId(obj), + getEntityDictionary().getIdType(relationshipType) + ), + obj)) + .collect(Collectors.toMap(Map.Entry::getKey, Map.Entry::getValue)); + } +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/SQLQuery.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/SQLQuery.java new file mode 100644 index 0000000000..624d13b02a --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/SQLQuery.java @@ -0,0 +1,53 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ + +package com.yahoo.elide.datastores.aggregation.queryengines.sql; + +import com.yahoo.elide.datastores.aggregation.query.Query; + +import lombok.Builder; +import lombok.Data; +import lombok.NonNull; + +/** + * Aids in constructing a SQL query from String fragments. + */ +@Data +@Builder +public class SQLQuery { + + private static final String SPACE = " "; + + @NonNull + private Query clientQuery; + + @NonNull + private String fromClause; + + @NonNull + private String projectionClause; + + @Builder.Default + private String joinClause = ""; + @Builder.Default + private String whereClause = ""; + @Builder.Default + private String groupByClause = ""; + @Builder.Default + private String havingClause = ""; + @Builder.Default + private String orderByClause = ""; + + @Override + public String toString() { + return String.format("SELECT %s FROM %s", projectionClause, fromClause) + + SPACE + joinClause + + SPACE + whereClause + + SPACE + groupByClause + + SPACE + havingClause + + SPACE + orderByClause; + } +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/SQLQueryConstructor.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/SQLQueryConstructor.java new file mode 100644 index 0000000000..e35ae1184c --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/SQLQueryConstructor.java @@ -0,0 +1,546 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.queryengines.sql; + +import static com.yahoo.elide.core.filter.FilterPredicate.appendAlias; +import static com.yahoo.elide.core.filter.FilterPredicate.getTypeAlias; +import static com.yahoo.elide.datastores.aggregation.queryengines.sql.SQLQueryEngine.generateColumnReference; +import static com.yahoo.elide.datastores.aggregation.queryengines.sql.SQLQueryEngine.getClassAlias; + +import com.yahoo.elide.core.EntityDictionary; +import com.yahoo.elide.core.Path; +import com.yahoo.elide.core.exceptions.InvalidPredicateException; +import com.yahoo.elide.core.filter.FilterPredicate; +import com.yahoo.elide.core.filter.FilterTranslator; +import com.yahoo.elide.core.filter.expression.FilterExpression; +import com.yahoo.elide.core.filter.expression.PredicateExtractionVisitor; +import com.yahoo.elide.core.sort.Sorting; +import com.yahoo.elide.datastores.aggregation.metadata.metric.MetricFunctionInvocation; +import com.yahoo.elide.datastores.aggregation.metadata.models.Table; +import com.yahoo.elide.datastores.aggregation.metadata.models.TimeDimension; +import com.yahoo.elide.datastores.aggregation.metadata.models.TimeDimensionGrain; +import com.yahoo.elide.datastores.aggregation.query.ColumnProjection; +import com.yahoo.elide.datastores.aggregation.query.Query; +import com.yahoo.elide.datastores.aggregation.query.TimeDimensionProjection; +import com.yahoo.elide.datastores.aggregation.queryengines.sql.annotation.FromSubquery; +import com.yahoo.elide.datastores.aggregation.queryengines.sql.annotation.FromTable; +import com.yahoo.elide.datastores.aggregation.queryengines.sql.annotation.JoinTo; +import com.yahoo.elide.datastores.aggregation.queryengines.sql.metadata.SQLAnalyticView; +import com.yahoo.elide.datastores.aggregation.queryengines.sql.metadata.SQLColumn; +import com.yahoo.elide.datastores.aggregation.queryengines.sql.query.SQLQueryTemplate; + +import org.hibernate.annotations.Subselect; + +import java.util.Collection; +import java.util.HashSet; +import java.util.LinkedHashSet; +import java.util.List; +import java.util.Map; +import java.util.Set; +import java.util.function.Function; +import java.util.stream.Collectors; +import java.util.stream.Stream; + +/** + * Class to construct query template into real sql query. + */ +public class SQLQueryConstructor { + private final EntityDictionary dictionary; + + public SQLQueryConstructor(EntityDictionary dictionary) { + this.dictionary = dictionary; + } + + /** + * Construct sql query with a template and sorting, where and having clause. + * + * @param clientQuery original query object + * @param template query template constructed from client query + * @param sorting sorting clause + * @param whereClause where clause + * @param havingClause having clause + * @return constructed SQLQuery object contains all information above + */ + public SQLQuery resolveTemplate(Query clientQuery, + SQLQueryTemplate template, + Sorting sorting, + FilterExpression whereClause, + FilterExpression havingClause) { + SQLAnalyticView queriedTable = (SQLAnalyticView) clientQuery.getAnalyticView(); + Class tableCls = clientQuery.getAnalyticView().getCls(); + String tableAlias = getClassAlias(tableCls); + + SQLQuery.SQLQueryBuilder builder = SQLQuery.builder().clientQuery(clientQuery); + + Set joinPaths = new HashSet<>(); + + String tableStatement = tableCls.isAnnotationPresent(FromSubquery.class) + ? "(" + tableCls.getAnnotation(FromSubquery.class).sql() + ")" + : tableCls.isAnnotationPresent(FromTable.class) + ? tableCls.getAnnotation(FromTable.class).name() + : queriedTable.getName(); + + builder.fromClause(String.format("%s AS %s", tableStatement, tableAlias)); + + builder.projectionClause(constructProjectionWithReference(template, queriedTable)); + + Set groupByDimensions = template.getGroupByDimensions(); + + if (!groupByDimensions.isEmpty()) { + if (!clientQuery.getMetrics().isEmpty()) { + builder.groupByClause(constructGroupByWithReference(groupByDimensions, queriedTable)); + } + + joinPaths.addAll(extractJoinPaths(groupByDimensions, queriedTable)); + } + + if (whereClause != null) { + builder.whereClause("WHERE " + translateFilterExpression(whereClause, this::generatePredicateReference)); + + joinPaths.addAll(extractJoinPaths(whereClause)); + } + + if (havingClause != null) { + builder.havingClause("HAVING " + translateFilterExpression( + havingClause, + (predicate) -> constructHavingClauseWithReference(predicate, queriedTable, template))); + + joinPaths.addAll(extractJoinPaths(havingClause)); + } + + if (sorting != null) { + Map sortClauses = sorting.getValidSortingRules(tableCls, dictionary); + builder.orderByClause(extractOrderBy(sortClauses, template)); + + joinPaths.addAll(extractJoinPaths(sortClauses)); + } + + builder.joinClause(extractJoin(joinPaths)); + + return builder.build(); + } + + /** + * Construct directly projection GROUP BY clause using column reference. + * + * @param groupByDimensions columns to project out + * @param queriedTable queried analytic view + * @return GROUP BY tb1.col1, tb2.col2, ... + */ + private String constructGroupByWithReference(Set groupByDimensions, + SQLAnalyticView queriedTable) { + return "GROUP BY " + groupByDimensions.stream() + .map(dimension -> resolveSQLColumnReference(dimension, queriedTable)) + .collect(Collectors.joining(", ")); + } + + /** + * Construct HAVING clause filter using physical column references. Metric fields need to be aggregated in HAVING. + * + * @param predicate a filter predicate in HAVING clause + * @param table Elide logical table this query is querying + * @param template query template + * @return an filter/constraint expression that can be put in HAVING clause + */ + private String constructHavingClauseWithReference(FilterPredicate predicate, + Table table, + SQLQueryTemplate template) { + Path.PathElement last = predicate.getPath().lastElement().get(); + Class lastClass = last.getType(); + String fieldName = last.getFieldName(); + + if (!lastClass.equals(table.getCls())) { + throw new InvalidPredicateException("The having clause can only reference fact table aggregations."); + } + + MetricFunctionInvocation metric = template.getMetrics().stream() + // TODO: filter predicate should support alias + .filter(invocation -> invocation.getAlias().equals(fieldName)) + .findFirst() + .orElse(null); + + if (metric != null) { + return metric.getFunctionExpression(); + } else { + return generatePredicateReference(predicate); + } + } + + /** + * Construct SELECT statement expression with metrics and dimensions directly using physical table column + * references. + * + * @param template query template with nested subquery + * @param queriedTable queried analytic view + * @return SELECT function(metric1) AS alias1, tb1.dimension1 AS alias2 + */ + private String constructProjectionWithReference(SQLQueryTemplate template, SQLAnalyticView queriedTable) { + // TODO: project metric field using table column reference + List metricProjections = template.getMetrics().stream() + .map(invocation -> invocation.getFunctionExpression() + " AS " + invocation.getAlias()) + .collect(Collectors.toList()); + + Class tableClass = queriedTable.getCls(); + + List dimensionProjections = template.getGroupByDimensions().stream() + .map(dimension -> { + String fieldName = dimension.getColumn().getName(); + + // relation to Non-JPA Entities object can't be projected + if (dictionary.isRelation(tableClass, fieldName)) { + Class relationshipClass = dictionary.getParameterizedType(tableClass, fieldName); + if (!dictionary.isJPAEntity(relationshipClass)) { + throw new InvalidPredicateException( + "Can't query on non-JPA relationship field: " + dimension.getColumn().getName()); + } + } + + return resolveSQLColumnReference(dimension, queriedTable) + " AS " + dimension.getAlias(); + }) + .collect(Collectors.toList()); + + if (metricProjections.isEmpty()) { + return "DISTINCT " + String.join(",", dimensionProjections); + } + + return Stream.concat(metricProjections.stream(), dimensionProjections.stream()) + .collect(Collectors.joining(",")); + } + + /** + * Build full join clause for all join paths. + * + * @param joinPaths paths that require joins + * @return built join clause that contains all needed relationship dimension joins for this query. + */ + private String extractJoin(Set joinPaths) { + Set joinClauses = new LinkedHashSet<>(); + + joinPaths.forEach(path -> addJoinClauses(path, joinClauses)); + + return String.join(" ", joinClauses); + } + + /** + * Add a join clause to a set of join clauses. + * + * @param joinPath join path + * @param alreadyJoined A set of joins that have already been computed. + */ + private void addJoinClauses(Path joinPath, Set alreadyJoined) { + String parentAlias = getTypeAlias(joinPath.getPathElements().get(0).getType()); + + for (Path.PathElement pathElement : joinPath.getPathElements()) { + String fieldName = pathElement.getFieldName(); + Class parentClass = pathElement.getType(); + + // Nothing left to join. + if (! dictionary.isRelation(parentClass, fieldName)) { + return; + } + + String joinFragment = extractJoinClause( + parentClass, + parentAlias, + pathElement.getFieldType(), + fieldName); + + alreadyJoined.add(joinFragment); + + parentAlias = appendAlias(parentAlias, fieldName); + } + } + + /** + * Build a single dimension join clause for joining a relationship table to the parent table. + * + * @param parentClass parent class + * @param parentAlias parent table alias + * @param relationshipClass relationship class + * @param relationshipName relationship field name + * @return built join clause i.e. LEFT JOIN table1 AS dimension1 ON table0.dim_id = dimension1.id + */ + private String extractJoinClause(Class parentClass, + String parentAlias, + Class relationshipClass, + String relationshipName) { + //TODO - support composite join keys. + //TODO - support joins where either side owns the relationship. + //TODO - Support INNER and RIGHT joins. + //TODO - Support toMany joins. + + String relationshipAlias = appendAlias(parentAlias, relationshipName); + String relationshipColumnName = dictionary.getAnnotatedColumnName(parentClass, relationshipName); + + // resolve the right hand side of JOIN + String joinSource = constructTableOrSubselect(relationshipClass); + + JoinTo joinTo = dictionary.getAttributeOrRelationAnnotation( + parentClass, + JoinTo.class, + relationshipColumnName); + + String joinClause = joinTo == null + ? String.format("%s.%s = %s.%s", + parentAlias, + relationshipColumnName, + relationshipAlias, + dictionary.getAnnotatedColumnName( + relationshipClass, + dictionary.getIdFieldName(relationshipClass))) + : extractJoinExpression(joinTo.joinClause(), parentAlias, relationshipAlias); + + return String.format("LEFT JOIN %s AS %s ON %s", + joinSource, + relationshipAlias, + joinClause); + } + + + /** + * Make a select statement for a table a sub select query. + * + * @param cls entity class + * @return tableName or (subselect query) + */ + private String constructTableOrSubselect(Class cls) { + return isSubselect(cls) + ? "(" + resolveTableOrSubselect(dictionary, cls) + ")" + : resolveTableOrSubselect(dictionary, cls); + } + + /** + * Given a list of columns to sort on, constructs an ORDER BY clause in SQL. + * @param sortClauses The list of sort columns and their sort order (ascending or descending). + * @return A SQL expression + */ + private String extractOrderBy(Map sortClauses, SQLQueryTemplate template) { + if (sortClauses.isEmpty()) { + return ""; + } + + //TODO - Ensure that order by columns are also present in the group by. + + return " ORDER BY " + sortClauses.entrySet().stream() + .map((entry) -> { + Path expandedPath = expandJoinToPath(entry.getKey()); + Sorting.SortOrder order = entry.getValue(); + + Path.PathElement last = expandedPath.lastElement().get(); + + MetricFunctionInvocation metric = template.getMetrics().stream() + // TODO: filter predicate should support alias + .filter(invocation -> invocation.getAlias().equals(last.getFieldName())) + .findFirst() + .orElse(null); + + String orderByClause = metric == null + ? generateColumnReference(expandedPath, dictionary) + : metric.getFunctionExpression(); + + return orderByClause + (order.equals(Sorting.SortOrder.desc) ? " DESC" : " ASC"); + }) + .collect(Collectors.joining(",")); + } + + /** + * Expands a predicate path (from a sort or filter predicate) to the path contained in + * the JoinTo annotation. If no JoinTo annotation is present, the original path is returned. + * + * @param path The path to expand. + * @return The expanded path. + */ + private Path expandJoinToPath(Path path) { + Path.PathElement pathRoot = path.getPathElements().get(0); + + Class entityClass = pathRoot.getType(); + String fieldName = pathRoot.getFieldName(); + + JoinTo joinTo = dictionary.getAttributeOrRelationAnnotation(entityClass, JoinTo.class, fieldName); + + if (joinTo == null || joinTo.path().equals("")) { + return path; + } + + return new Path(entityClass, dictionary, joinTo.path()); + } + + /** + * Given a filter expression, extracts any entity relationship traversals that require joins. + * + * @param expression The filter expression + * @return A set of path elements that capture a relationship traversal. + */ + private Set extractJoinPaths(FilterExpression expression) { + Collection predicates = expression.accept(new PredicateExtractionVisitor()); + + return predicates.stream() + .map(FilterPredicate::getPath) + .map(this::expandJoinToPath) + .filter(path -> path.getPathElements().size() > 1) + .collect(Collectors.toCollection(LinkedHashSet::new)); + } + + /** + * Given a list of columns to sort on, extracts any entity relationship traversals that require joins. + * + * @param sortClauses The list of sort columns and their sort order (ascending or descending). + * @return A set of path elements that capture a relationship traversal. + */ + private Set extractJoinPaths(Map sortClauses) { + return sortClauses.keySet().stream() + .map(this::expandJoinToPath) + .collect(Collectors.toCollection(LinkedHashSet::new)); + } + + /** + * Given the set of group by dimensions, extract any entity relationship traversals that require joins. + * This method takes in a {@link SQLAnalyticView} because the sql join path meta data is stored in it. + * + * @param groupByDimensions The list of dimensions we are grouping on. + * @param queriedTable queried analytic view + * @return A set of path elements that capture a relationship traversal. + */ + private Set extractJoinPaths(Set groupByDimensions, + SQLAnalyticView queriedTable) { + return resolveSQLColumns(groupByDimensions, queriedTable).stream() + .filter((dim) -> dim.getJoinPath() != null) + .map(SQLColumn::getJoinPath) + .collect(Collectors.toCollection(LinkedHashSet::new)); + } + + /** + * Translates a filter expression into SQL. + * + * @param expression The filter expression + * @param columnGenerator A function which generates a column reference in SQL from a FilterPredicate. + * @return A SQL expression + */ + private String translateFilterExpression(FilterExpression expression, + Function columnGenerator) { + FilterTranslator filterVisitor = new FilterTranslator(); + + return filterVisitor.apply(expression, columnGenerator); + } + + /** + * Converts a filter predicate into a SQL WHERE/HAVING clause column reference. + * + * @param predicate The predicate to convert + * @return A SQL fragment that references a database column + */ + private String generatePredicateReference(FilterPredicate predicate) { + return generateColumnReference(predicate.getPath(), dictionary); + } + + /** + * Resolve all projected sql column from a queried table. + * + * @param columnProjections projections + * @param queriedTable sql analytic view + * @return projected columns + */ + private Set resolveSQLColumns(Set columnProjections, SQLAnalyticView queriedTable) { + return columnProjections.stream() + .map(colProjection -> queriedTable.getColumn(colProjection.getColumn().getName())) + .collect(Collectors.toSet()); + } + + /** + * Resolve projected sql column as column reference from a queried table. + * If the projection is {@link TimeDimensionProjection}, the correct time grain expression would be used. + * + * @param columnProjection projection + * @param queriedTable sql analytic view + * @return projected columns + */ + private String resolveSQLColumnReference(ColumnProjection columnProjection, SQLAnalyticView queriedTable) { + SQLColumn sqlColumn = queriedTable.getColumn(columnProjection.getColumn().getName()); + + if (columnProjection instanceof TimeDimensionProjection) { + TimeDimension timeDimension = ((TimeDimensionProjection) columnProjection).getTimeDimension(); + TimeDimensionGrain grainInfo = timeDimension.getSupportedGrains().stream() + .filter(g -> g.getGrain().equals(((TimeDimensionProjection) columnProjection).getGrain())) + .findFirst() + .orElseThrow(() -> new IllegalStateException("Requested time grain not supported.")); + + //TODO - We will likely migrate to a templating language when we support parameterized metrics. + return String.format(grainInfo.getExpression(), sqlColumn.getReference()); + } else { + return sqlColumn.getReference(); + } + } + + /** + * Maps an entity class to a physical table of subselect query, if neither {@link javax.persistence.Table} + * nor {@link Subselect} annotation is present on this class, try {@link FromTable} and {@link FromSubquery}. + * + * @param cls The entity class. + * @return The physical SQL table or subselect query. + */ + private static String resolveTableOrSubselect(EntityDictionary dictionary, Class cls) { + if (isSubselect(cls)) { + if (cls.isAnnotationPresent(FromSubquery.class)) { + return dictionary.getAnnotation(cls, FromSubquery.class).sql(); + } else { + return dictionary.getAnnotation(cls, Subselect.class).value(); + } + } else { + javax.persistence.Table table = dictionary.getAnnotation(cls, javax.persistence.Table.class); + + if (table != null) { + return resolveTableAnnotation(table); + } else { + FromTable fromTable = dictionary.getAnnotation(cls, FromTable.class); + + return fromTable != null ? fromTable.name() : dictionary.getJsonAliasFor(cls); + } + } + } + + /** + * Get the full table name from JPA {@link javax.persistence.Table} annotation. + * + * @param table table annotation + * @return catalog.schema.name + */ + private static String resolveTableAnnotation(javax.persistence.Table table) { + StringBuilder fullTableName = new StringBuilder(); + + if (!"".equals(table.catalog())) { + fullTableName.append(table.catalog()).append("."); + } + if (!"".equals(table.schema())) { + fullTableName.append(table.schema()).append("."); + } + fullTableName.append(table.name()); + + return fullTableName.toString(); + } + + /** + * Construct a join on clause based on given constraint expression, replace "%from" with from table alias + * and "%join" with join table alias. + * + * @param joinClause sql join constraint + * @param fromAlias from table alias + * @param joinToAlias join to table alias + * @return sql string that represents a full join condition + */ + private String extractJoinExpression(String joinClause, String fromAlias, String joinToAlias) { + return joinClause.replace("%from", fromAlias).replace("%join", joinToAlias); + } + + /** + * Check whether a class is mapped to a subselect query instead of a physical table. + * + * @param cls The entity class + * @return True if the class has {@link Subselect} annotation + */ + private static boolean isSubselect(Class cls) { + return cls.isAnnotationPresent(Subselect.class) || cls.isAnnotationPresent(FromSubquery.class); + } +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/SQLQueryEngine.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/SQLQueryEngine.java new file mode 100644 index 0000000000..b4404131a8 --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/SQLQueryEngine.java @@ -0,0 +1,299 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.queryengines.sql; + +import static com.yahoo.elide.core.filter.FilterPredicate.getPathAlias; + +import com.yahoo.elide.core.EntityDictionary; +import com.yahoo.elide.core.Path; +import com.yahoo.elide.core.TimedFunction; +import com.yahoo.elide.core.exceptions.InvalidPredicateException; +import com.yahoo.elide.core.filter.FilterPredicate; +import com.yahoo.elide.core.filter.expression.PredicateExtractionVisitor; +import com.yahoo.elide.core.pagination.Pagination; +import com.yahoo.elide.datastores.aggregation.QueryEngine; +import com.yahoo.elide.datastores.aggregation.metadata.MetaDataStore; +import com.yahoo.elide.datastores.aggregation.metadata.metric.MetricFunctionInvocation; +import com.yahoo.elide.datastores.aggregation.metadata.models.AnalyticView; +import com.yahoo.elide.datastores.aggregation.metadata.models.MetricFunction; +import com.yahoo.elide.datastores.aggregation.metadata.models.Table; +import com.yahoo.elide.datastores.aggregation.query.ColumnProjection; +import com.yahoo.elide.datastores.aggregation.query.Query; +import com.yahoo.elide.datastores.aggregation.query.TimeDimensionProjection; +import com.yahoo.elide.datastores.aggregation.queryengines.sql.annotation.JoinTo; +import com.yahoo.elide.datastores.aggregation.queryengines.sql.metadata.SQLAnalyticView; +import com.yahoo.elide.datastores.aggregation.queryengines.sql.metadata.SQLColumn; +import com.yahoo.elide.datastores.aggregation.queryengines.sql.metadata.SQLTable; +import com.yahoo.elide.datastores.aggregation.queryengines.sql.metric.SQLMetricFunction; +import com.yahoo.elide.datastores.aggregation.queryengines.sql.query.SQLQueryTemplate; +import com.yahoo.elide.utils.coerce.CoerceUtil; + +import org.hibernate.jpa.QueryHints; + +import lombok.Getter; +import lombok.extern.slf4j.Slf4j; + +import java.util.ArrayList; +import java.util.Collection; +import java.util.Collections; +import java.util.LinkedHashSet; +import java.util.List; +import java.util.Map; +import java.util.Set; +import java.util.function.Function; +import java.util.stream.Collectors; +import javax.persistence.EntityManager; +import javax.persistence.EntityManagerFactory; +import javax.persistence.EntityTransaction; + +/** + * QueryEngine for SQL backed stores. + */ +@Slf4j +public class SQLQueryEngine implements QueryEngine { + + private final EntityManagerFactory emf; + private final EntityDictionary metadataDictionary; + + @Getter + private Map, Table> tables; + + public SQLQueryEngine(EntityManagerFactory emf, MetaDataStore metaDataStore) { + this.emf = emf; + this.metadataDictionary = metaDataStore.getDictionary(); + + Set tables = metaDataStore.getMetaData(Table.class); + tables.addAll(metaDataStore.getMetaData(AnalyticView.class)); + + this.tables = tables.stream() + .map(table -> table instanceof AnalyticView + ? new SQLAnalyticView(table.getCls(), metadataDictionary) + : new SQLTable(table.getCls(), metadataDictionary)) + .collect(Collectors.toMap(Table::getCls, Function.identity())); + } + + @Override + public Table getTable(Class entityClass) { + return tables.get(entityClass); + } + + @Override + public Iterable executeQuery(Query query) { + EntityManager entityManager = null; + EntityTransaction transaction = null; + try { + entityManager = emf.createEntityManager(); + + // manually begin the transaction + transaction = entityManager.getTransaction(); + if (!transaction.isActive()) { + transaction.begin(); + } + + // Translate the query into SQL. + SQLQuery sql = toSQL(query); + log.debug("SQL Query: " + sql); + + javax.persistence.Query jpaQuery = entityManager.createNativeQuery(sql.toString()); + + Pagination pagination = query.getPagination(); + if (pagination != null) { + jpaQuery.setFirstResult(pagination.getOffset()); + jpaQuery.setMaxResults(pagination.getLimit()); + + if (pagination.isGenerateTotals()) { + + SQLQuery paginationSQL = toPageTotalSQL(sql); + javax.persistence.Query pageTotalQuery = + entityManager.createNativeQuery(paginationSQL.toString()) + .setHint(QueryHints.HINT_READONLY, true); + + //Supply the query parameters to the query + supplyFilterQueryParameters(query, pageTotalQuery); + + //Run the Pagination query and log the time spent. + long total = new TimedFunction<>( + () -> CoerceUtil.coerce(pageTotalQuery.getSingleResult(), Long.class), + "Running Query: " + paginationSQL + ).get(); + + pagination.setPageTotals(total); + } + } + + // Supply the query parameters to the query + supplyFilterQueryParameters(query, jpaQuery); + + // Run the primary query and log the time spent. + List results = new TimedFunction<>( + () -> jpaQuery.setHint(QueryHints.HINT_READONLY, true).getResultList(), + "Running Query: " + sql).get(); + + return new SQLEntityHydrator(results, query, metadataDictionary, entityManager).hydrate(); + } finally { + if (transaction != null && transaction.isActive()) { + transaction.commit(); + } + if (entityManager != null) { + entityManager.close(); + } + } + } + + /** + * Translates the client query into SQL. + * + * @param query the client query. + * @return the SQL query. + */ + protected SQLQuery toSQL(Query query) { + Set groupByDimensions = new LinkedHashSet<>(query.getGroupByDimensions()); + Set timeDimensions = new LinkedHashSet<>(query.getTimeDimensions()); + + // TODO: handle the case of more than one time dimensions + TimeDimensionProjection timeDimension = timeDimensions.stream().findFirst().orElse(null); + + SQLQueryTemplate queryTemplate = query.getMetrics().stream() + .map(invocation -> { + MetricFunction function = invocation.getFunction(); + + if (!(function instanceof SQLMetricFunction)) { + throw new InvalidPredicateException("Non-SQL metric function on " + invocation.getAlias()); + } + + return ((SQLMetricFunction) function).resolve( + invocation.getArgumentMap(), + invocation.getAlias(), + groupByDimensions, + timeDimension); + }) + .reduce(SQLQueryTemplate::merge) + .orElse(new SQLQueryTemplate() { + @Override + public List getMetrics() { + return Collections.emptyList(); + } + + @Override + public Set getNonTimeDimensions() { + return groupByDimensions; + } + + @Override + public TimeDimensionProjection getTimeDimension() { + return timeDimension; + } + }); + + return new SQLQueryConstructor(metadataDictionary).resolveTemplate( + query, + queryTemplate, + query.getSorting(), + query.getWhereFilter(), + query.getHavingFilter()); + } + + + /** + * Given a JPA query, replaces any parameters with their values from client query. + * + * @param query The client query + * @param jpaQuery The JPA query + */ + private void supplyFilterQueryParameters(Query query, + javax.persistence.Query jpaQuery) { + + Collection predicates = new ArrayList<>(); + if (query.getWhereFilter() != null) { + predicates.addAll(query.getWhereFilter().accept(new PredicateExtractionVisitor())); + } + + if (query.getHavingFilter() != null) { + predicates.addAll(query.getHavingFilter().accept(new PredicateExtractionVisitor())); + } + + for (FilterPredicate filterPredicate : predicates) { + if (filterPredicate.getOperator().isParameterized()) { + boolean shouldEscape = filterPredicate.isMatchingOperator(); + filterPredicate.getParameters().forEach(param -> { + jpaQuery.setParameter(param.getName(), shouldEscape ? param.escapeMatching() : param.getValue()); + }); + } + } + } + + /** + * Takes a SQLQuery and creates a new clone that instead returns the total number of records of the original + * query. + * + * @param sql The original query + * @return A new query that returns the total number of records. + */ + private SQLQuery toPageTotalSQL(SQLQuery sql) { + // TODO: refactor this method + String groupByDimensions = + extractSQLDimensions(sql.getClientQuery(), (SQLAnalyticView) sql.getClientQuery().getAnalyticView()) + .stream() + .map(SQLColumn::getReference) + .collect(Collectors.joining(", ")); + + String projectionClause = String.format("COUNT(DISTINCT(%s))", groupByDimensions); + + return SQLQuery.builder() + .clientQuery(sql.getClientQuery()) + .projectionClause(projectionClause) + .fromClause(sql.getFromClause()) + .joinClause(sql.getJoinClause()) + .whereClause(sql.getWhereClause()) + .build(); + } + + /** + * Extract dimension projects in a query to sql dimensions. + * + * @param query requested query + * @param queriedTable queried analytic view + * @return sql dimensions in this query + */ + private List extractSQLDimensions(Query query, SQLAnalyticView queriedTable) { + return query.getDimensions().stream() + .map(projection -> queriedTable.getColumn(projection.getColumn().getName())) + .collect(Collectors.toList()); + } + + /** + * Converts a filter predicate path into a SQL column reference. + * All other code should use this method to generate sql column reference, no matter where the reference is used ( + * select statement, group by clause, where clause, having clause or order by clause). + * + * @param path The predicate path to convert + * @param dictionary dictionary to expand joinTo path + * @return A SQL fragment that references a database column + */ + public static String generateColumnReference(Path path, EntityDictionary dictionary) { + Path.PathElement last = path.lastElement().get(); + Class lastClass = last.getType(); + String fieldName = last.getFieldName(); + + JoinTo joinTo = dictionary.getAttributeOrRelationAnnotation(lastClass, JoinTo.class, fieldName); + + if (joinTo == null) { + return getPathAlias(path) + "." + dictionary.getAnnotatedColumnName(lastClass, last.getFieldName()); + } else { + return generateColumnReference(new Path(lastClass, dictionary, joinTo.path()), dictionary); + } + } + + /** + * Get alias for an entity class. + * + * @param entityClass entity class + * @return alias + */ + public static String getClassAlias(Class entityClass) { + return FilterPredicate.getTypeAlias(entityClass); + } +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/SQLQueryEngineFactory.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/SQLQueryEngineFactory.java new file mode 100644 index 0000000000..c7834b0bc3 --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/SQLQueryEngineFactory.java @@ -0,0 +1,30 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.queryengines.sql; + +import com.yahoo.elide.datastores.aggregation.QueryEngine; +import com.yahoo.elide.datastores.aggregation.QueryEngineFactory; +import com.yahoo.elide.datastores.aggregation.metadata.MetaDataStore; +import lombok.Getter; + +import javax.persistence.EntityManagerFactory; + +/** + * Object that constructs {@link QueryEngine} based on given entityDictionary and entityManagerFactory. + */ +public class SQLQueryEngineFactory implements QueryEngineFactory { + @Getter + private EntityManagerFactory emf; + + public SQLQueryEngineFactory(EntityManagerFactory emf) { + this.emf = emf; + } + + @Override + public QueryEngine buildQueryEngine(MetaDataStore metaDataStore) { + return new SQLQueryEngine(emf, metaDataStore); + } +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/annotation/FromSubquery.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/annotation/FromSubquery.java new file mode 100644 index 0000000000..9145416051 --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/annotation/FromSubquery.java @@ -0,0 +1,28 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.queryengines.sql.annotation; + +import java.lang.annotation.Documented; +import java.lang.annotation.ElementType; +import java.lang.annotation.Retention; +import java.lang.annotation.RetentionPolicy; +import java.lang.annotation.Target; + +/** + * Indicates that the entity or field is derived from a native SQL subquery. + */ +@Documented +@Target({ElementType.TYPE, ElementType.METHOD, ElementType.FIELD}) +@Retention(RetentionPolicy.RUNTIME) +public @interface FromSubquery { + + /** + * The SQL subquery. + * + * @return The SQL subquery. + */ + String sql(); +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/annotation/FromTable.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/annotation/FromTable.java new file mode 100644 index 0000000000..4715db7639 --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/annotation/FromTable.java @@ -0,0 +1,28 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.queryengines.sql.annotation; + +import java.lang.annotation.Documented; +import java.lang.annotation.ElementType; +import java.lang.annotation.Retention; +import java.lang.annotation.RetentionPolicy; +import java.lang.annotation.Target; + +/** + * Indicates that the entity is derived directly from a physical table or view in the database. + */ +@Documented +@Target({ElementType.TYPE}) +@Retention(RetentionPolicy.RUNTIME) +public @interface FromTable { + + /** + * The table or view name. + * + * @return The table or view name. + */ + String name(); +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/annotation/JoinTo.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/annotation/JoinTo.java new file mode 100644 index 0000000000..299135aea0 --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/annotation/JoinTo.java @@ -0,0 +1,37 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.queryengines.sql.annotation; + +import java.lang.annotation.Documented; +import java.lang.annotation.ElementType; +import java.lang.annotation.Retention; +import java.lang.annotation.RetentionPolicy; +import java.lang.annotation.Target; + +/** + * Indicates that the annotated entity field is derived from a join to another table. + * This annotation must be present for relationship to views. + */ +@Documented +@Target({ElementType.FIELD, ElementType.METHOD}) +@Retention(RetentionPolicy.RUNTIME) +public @interface JoinTo { + + /** + * Dot separated path through the entity relationship graph to an attribute. + * If the current entity is author, then a path would be "book.publisher.name". + * @return The path + */ + String path() default ""; + + /** + * Join on clause constraint for customizing relationship joins as a plain sql string. Provided in the model. + * Use "%from" and "%join% to represent the two sides of join. + * + * @return join constraint like %from.col1 = %join.col2 + */ + String joinClause() default ""; +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metadata/SQLAnalyticView.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metadata/SQLAnalyticView.java new file mode 100644 index 0000000000..89d5edfbad --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metadata/SQLAnalyticView.java @@ -0,0 +1,34 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.queryengines.sql.metadata; + +import static com.yahoo.elide.datastores.aggregation.queryengines.sql.metadata.SQLTable.resolveSQLDimensions; + +import com.yahoo.elide.core.EntityDictionary; +import com.yahoo.elide.datastores.aggregation.metadata.models.AnalyticView; + +import lombok.Data; +import lombok.EqualsAndHashCode; + +import java.util.Map; + +/** + * SQL extension of {@link AnalyticView} which also contains sql column meta data. + */ +@EqualsAndHashCode(callSuper = true) +@Data +public class SQLAnalyticView extends AnalyticView { + private Map sqlColumns; + + public SQLAnalyticView(Class cls, EntityDictionary dictionary) { + super(cls, dictionary); + this.sqlColumns = resolveSQLDimensions(cls, dictionary); + } + + public SQLColumn getColumn(String fieldName) { + return sqlColumns.get(fieldName); + } +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metadata/SQLColumn.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metadata/SQLColumn.java new file mode 100644 index 0000000000..2eac1ea9a2 --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metadata/SQLColumn.java @@ -0,0 +1,42 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.queryengines.sql.metadata; + +import static com.yahoo.elide.datastores.aggregation.queryengines.sql.SQLQueryEngine.generateColumnReference; +import static com.yahoo.elide.datastores.aggregation.queryengines.sql.SQLQueryEngine.getClassAlias; + +import com.yahoo.elide.core.EntityDictionary; +import com.yahoo.elide.core.Path; +import com.yahoo.elide.datastores.aggregation.metadata.models.Column; +import com.yahoo.elide.datastores.aggregation.queryengines.sql.annotation.JoinTo; + +import lombok.Getter; + +/** + * SQLColumn contains meta data about underlying physical table. + */ +public class SQLColumn extends Column { + @Getter + private final String reference; + + @Getter + private final Path joinPath; + + protected SQLColumn(Class tableClass, String fieldName, EntityDictionary dictionary) { + super(tableClass, fieldName, dictionary); + + JoinTo joinTo = dictionary.getAttributeOrRelationAnnotation(tableClass, JoinTo.class, fieldName); + + if (joinTo == null || joinTo.path().equals("")) { + this.reference = getClassAlias(tableClass) + "." + dictionary.getAnnotatedColumnName(tableClass, fieldName); + this.joinPath = null; + } else { + Path path = new Path(tableClass, dictionary, joinTo.path()); + this.reference = generateColumnReference(path, dictionary); + this.joinPath = path; + } + } +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metadata/SQLTable.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metadata/SQLTable.java new file mode 100644 index 0000000000..f0e230cfc7 --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metadata/SQLTable.java @@ -0,0 +1,51 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.queryengines.sql.metadata; + +import static com.yahoo.elide.datastores.aggregation.metadata.MetaDataStore.isMetricField; + +import com.yahoo.elide.core.EntityDictionary; +import com.yahoo.elide.datastores.aggregation.metadata.models.Column; +import com.yahoo.elide.datastores.aggregation.metadata.models.Table; + +import lombok.Data; +import lombok.EqualsAndHashCode; + +import java.util.Map; +import java.util.function.Function; +import java.util.stream.Collectors; + +/** + * SQL extension of {@link Table} which also contains sql column meta data. + */ +@EqualsAndHashCode(callSuper = true) +@Data +public class SQLTable extends Table { + private Map sqlColumns; + + public SQLTable(Class cls, EntityDictionary dictionary) { + super(cls, dictionary); + this.sqlColumns = resolveSQLDimensions(cls, dictionary); + } + + /** + * Resolve all sql columns of a table. + * + * @param cls table class + * @param dictionary dictionary contains the table class + * @return all resolved sql column metadata + */ + public static Map resolveSQLDimensions(Class cls, EntityDictionary dictionary) { + return dictionary.getAllFields(cls).stream() + .filter(field -> Column.getDataType(cls, field, dictionary) != null) + .filter(field -> !isMetricField(dictionary, cls, field)) + .collect(Collectors.toMap(Function.identity(), field -> new SQLColumn(cls, field, dictionary))); + } + + public SQLColumn getColumn(String fieldName) { + return sqlColumns.get(fieldName); + } +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metric/SQLMetricFunction.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metric/SQLMetricFunction.java new file mode 100644 index 0000000000..f8b633416b --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metric/SQLMetricFunction.java @@ -0,0 +1,62 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.queryengines.sql.metric; + +import com.yahoo.elide.datastores.aggregation.metadata.metric.MetricFunctionInvocation; +import com.yahoo.elide.datastores.aggregation.metadata.models.FunctionArgument; +import com.yahoo.elide.datastores.aggregation.metadata.models.MetricFunction; +import com.yahoo.elide.datastores.aggregation.query.ColumnProjection; +import com.yahoo.elide.datastores.aggregation.query.TimeDimensionProjection; +import com.yahoo.elide.datastores.aggregation.queryengines.sql.query.SQLQueryTemplate; +import com.yahoo.elide.request.Argument; + +import java.util.Collections; +import java.util.List; +import java.util.Map; +import java.util.Set; + +/** + * SQL extension of {@link MetricFunction} which would be invoked as sql and can construct sql templates. + */ +public class SQLMetricFunction extends MetricFunction { + public SQLMetricFunction(String name, String longName, String description, String expression, + Set arguments) { + super(name, longName, description, expression, arguments); + } + + /** + * Construct a sql query template for a physical table with provided information. + * Table name would be filled in when convert the template into real query. + * + * @param arguments arguments provided in the request + * @param alias result alias + * @param dimensions groupBy dimensions + * @param timeDimension aggregated time dimension + * @return SELECT function(arguments, fields) AS alias GROUP BY dimensions, timeDimension + */ + public SQLQueryTemplate resolve(Map arguments, + String alias, + Set dimensions, + TimeDimensionProjection timeDimension) { + MetricFunctionInvocation invoked = invoke(arguments, alias); + return new SQLQueryTemplate() { + @Override + public List getMetrics() { + return Collections.singletonList(invoked); + } + + @Override + public Set getNonTimeDimensions() { + return dimensions; + } + + @Override + public TimeDimensionProjection getTimeDimension() { + return timeDimension; + } + }; + } +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metric/functions/SqlAvg.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metric/functions/SqlAvg.java new file mode 100644 index 0000000000..4ff7ddeb81 --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metric/functions/SqlAvg.java @@ -0,0 +1,19 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.queryengines.sql.metric.functions; + +import com.yahoo.elide.datastores.aggregation.queryengines.sql.metric.SQLMetricFunction; + +import java.util.Collections; + +/** + * Average of a field. + */ +public class SqlAvg extends SQLMetricFunction { + public SqlAvg() { + super("avg", "average", "sql average function", "AVG(%s)", Collections.emptySet()); + } +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metric/functions/SqlMax.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metric/functions/SqlMax.java new file mode 100644 index 0000000000..7a0918e927 --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metric/functions/SqlMax.java @@ -0,0 +1,19 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.queryengines.sql.metric.functions; + +import com.yahoo.elide.datastores.aggregation.queryengines.sql.metric.SQLMetricFunction; + +import java.util.Collections; + +/** + * Max of a field. + */ +public class SqlMax extends SQLMetricFunction { + public SqlMax() { + super("max", "max", "sql max function", "MAX(%s)", Collections.emptySet()); + } +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metric/functions/SqlMin.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metric/functions/SqlMin.java new file mode 100644 index 0000000000..221559e5fd --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metric/functions/SqlMin.java @@ -0,0 +1,19 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.queryengines.sql.metric.functions; + +import com.yahoo.elide.datastores.aggregation.queryengines.sql.metric.SQLMetricFunction; + +import java.util.Collections; + +/** + * Min of a field. + */ +public class SqlMin extends SQLMetricFunction { + public SqlMin() { + super("min", "min", "sql min function", "MIN(%s)", Collections.emptySet()); + } +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metric/functions/SqlSum.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metric/functions/SqlSum.java new file mode 100644 index 0000000000..9319f817bb --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metric/functions/SqlSum.java @@ -0,0 +1,19 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.queryengines.sql.metric.functions; + +import com.yahoo.elide.datastores.aggregation.queryengines.sql.metric.SQLMetricFunction; + +import java.util.Collections; + +/** + * Sum of a field. + */ +public class SqlSum extends SQLMetricFunction { + public SqlSum() { + super("sum", "sum", "sql sum function", "SUM(%s)", Collections.emptySet()); + } +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/query/SQLQueryTemplate.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/query/SQLQueryTemplate.java new file mode 100644 index 0000000000..efc06caecf --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/query/SQLQueryTemplate.java @@ -0,0 +1,111 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.queryengines.sql.query; + +import com.yahoo.elide.datastores.aggregation.metadata.metric.MetricFunctionInvocation; +import com.yahoo.elide.datastores.aggregation.query.ColumnProjection; +import com.yahoo.elide.datastores.aggregation.query.TimeDimensionProjection; +import com.yahoo.elide.datastores.aggregation.time.TimeGrain; + +import com.google.common.collect.Sets; + +import java.util.ArrayList; +import java.util.Collections; +import java.util.List; +import java.util.Set; + +/** + * SQLQueryTemplate contains projections information about a sql query. + */ +public interface SQLQueryTemplate { + /** + * Get all invoked metrics in this query. + * + * @return invoked metrics + */ + List getMetrics(); + + /** + * Get all non-time dimensions in this query. + * + * @return non-time dimensions + */ + Set getNonTimeDimensions(); + + /** + * Get aggregated time dimension for this query. + * + * @return time dimension + */ + TimeDimensionProjection getTimeDimension(); + + /** + * Get all GROUP BY dimensions in this query, include time and non-time dimensions. + * + * @return all GROUP BY dimensions + */ + default Set getGroupByDimensions() { + return getTimeDimension() == null + ? getNonTimeDimensions() + : Sets.union(getNonTimeDimensions(), Collections.singleton(getTimeDimension())); + } + + /** + * Get a copy of this query with a requested time grain. + * + * @param timeGrain requested time grain + * @return a copied query template + */ + default SQLQueryTemplate toTimeGrain(TimeGrain timeGrain) { + SQLQueryTemplate wrapped = this; + return new SQLQueryTemplate() { + @Override + public List getMetrics() { + return wrapped.getMetrics(); + } + + @Override + public Set getNonTimeDimensions() { + return wrapped.getNonTimeDimensions(); + } + + @Override + public TimeDimensionProjection getTimeDimension() { + return wrapped.getTimeDimension().toTimeGrain(timeGrain); + } + }; + } + + /** + * Merge with other query. + * + * @param second other query template + * @return merged query template + */ + default SQLQueryTemplate merge(SQLQueryTemplate second) { + SQLQueryTemplate first = this; + // TODO: validate dimension + List merged = new ArrayList<>(first.getMetrics()); + merged.addAll(second.getMetrics()); + + return new SQLQueryTemplate() { + @Override + public List getMetrics() { + return merged; + } + + @Override + public Set getNonTimeDimensions() { + return first.getNonTimeDimensions(); + } + + @Override + public TimeDimensionProjection getTimeDimension() { + return first.getTimeDimension(); + } + }; + } +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/time/TimeGrain.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/time/TimeGrain.java new file mode 100644 index 0000000000..c45102a3ac --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/time/TimeGrain.java @@ -0,0 +1,26 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.time; + +import java.time.Period; + +/** + * {@link TimeGrain} is a set of concrete {@link TimeGrain} implementations which support "natural" time buckets. + */ +public enum TimeGrain { + + DAY(Period.ofDays(1)), + WEEK(Period.ofWeeks(1)), + MONTH(Period.ofMonths(1)), + YEAR(Period.ofYears(1)) + ; + + private final Period period; + + TimeGrain(final Period period) { + this.period = period; + } +} diff --git a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/EntityProjectionTranslatorTest.java b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/EntityProjectionTranslatorTest.java new file mode 100644 index 0000000000..6078874b1e --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/EntityProjectionTranslatorTest.java @@ -0,0 +1,164 @@ +/* + * Copyright 2020, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation; + +import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.junit.jupiter.api.Assertions.assertThrows; + +import com.yahoo.elide.core.exceptions.InvalidOperationException; +import com.yahoo.elide.core.filter.dialect.ParseException; +import com.yahoo.elide.core.filter.expression.FilterExpression; +import com.yahoo.elide.datastores.aggregation.example.Country; +import com.yahoo.elide.datastores.aggregation.example.PlayerStats; +import com.yahoo.elide.datastores.aggregation.filter.visitor.FilterConstraints; +import com.yahoo.elide.datastores.aggregation.filter.visitor.SplitFilterExpressionVisitor; +import com.yahoo.elide.datastores.aggregation.framework.SQLUnitTest; +import com.yahoo.elide.datastores.aggregation.metadata.models.Table; +import com.yahoo.elide.datastores.aggregation.query.ColumnProjection; +import com.yahoo.elide.datastores.aggregation.query.Query; +import com.yahoo.elide.datastores.aggregation.query.TimeDimensionProjection; +import com.yahoo.elide.datastores.aggregation.time.TimeGrain; +import com.yahoo.elide.request.Argument; +import com.yahoo.elide.request.Attribute; +import com.yahoo.elide.request.EntityProjection; +import com.yahoo.elide.request.Relationship; + +import org.junit.jupiter.api.BeforeAll; +import org.junit.jupiter.api.Test; + +import java.util.ArrayList; +import java.util.Date; +import java.util.List; + +public class EntityProjectionTranslatorTest extends SQLUnitTest { + private static EntityProjection basicProjection = EntityProjection.builder() + .type(PlayerStats.class) + .attribute(Attribute.builder() + .type(long.class) + .name("lowScore") + .build()) + .attribute(Attribute.builder() + .type(String.class) + .name("overallRating") + .build()) + .relationship(Relationship.builder() + .name("country") + .projection(EntityProjection.builder() + .type(Country.class) + .attribute(Attribute.builder() + .type(String.class) + .name("name") + .build()) + .build()) + .build()) + .build(); + + @BeforeAll + public static void init() { + SQLUnitTest.init(); + } + + @Test + public void testBasicTranslation() { + EntityProjectionTranslator translator = new EntityProjectionTranslator( + playerStatsTable, + basicProjection, + dictionary + ); + + Query query = translator.getQuery(); + + assertEquals(playerStatsTable, query.getAnalyticView()); + assertEquals(1, query.getMetrics().size()); + assertEquals("lowScore", query.getMetrics().get(0).getAlias()); + assertEquals(2, query.getGroupByDimensions().size()); + + List dimensions = new ArrayList<>(query.getGroupByDimensions()); + assertEquals("overallRating", dimensions.get(0).getColumn().getName()); + assertEquals("country", dimensions.get(1).getColumn().getName()); + } + + @Test + public void testWherePromotion() throws ParseException { + FilterExpression originalFilter = filterParser.parseFilterExpression("overallRating==Good,lowScore<45", + PlayerStats.class, false); + + EntityProjection projection = basicProjection.copyOf() + .filterExpression(originalFilter) + .build(); + + EntityProjectionTranslator translator = new EntityProjectionTranslator( + playerStatsTable, + projection, + dictionary + ); + + Query query = translator.getQuery(); + + SplitFilterExpressionVisitor visitor = new SplitFilterExpressionVisitor(playerStatsTable); + FilterConstraints constraints = originalFilter.accept(visitor); + FilterExpression whereFilter = constraints.getWhereExpression(); + FilterExpression havingFilter = constraints.getHavingExpression(); + assertEquals(whereFilter, query.getWhereFilter()); + assertEquals(havingFilter, query.getHavingFilter()); + } + + @Test + public void testInvalidQueriedTable() { + EntityProjection projection = EntityProjection.builder() + .type(Country.class) + .build(); + + assertThrows(InvalidOperationException.class, () -> new EntityProjectionTranslator( + new Table(Country.class, dictionary), + projection, + dictionary + )); + } + + @Test + public void testTimeDimension() { + EntityProjection projection = basicProjection.copyOf() + .attribute(Attribute.builder() + .type(Date.class) + .name("recordedDate") + .build()) + .build(); + + EntityProjectionTranslator translator = new EntityProjectionTranslator( + playerStatsTable, + projection, + dictionary + ); + + Query query = translator.getQuery(); + + List timeDimensions = new ArrayList<>(query.getTimeDimensions()); + assertEquals(1, timeDimensions.size()); + assertEquals("recordedDate", timeDimensions.get(0).getAlias()); + assertEquals(TimeGrain.DAY, timeDimensions.get(0).getGrain()); + } + + @Test + public void testUnsupportedTimeGrain() { + EntityProjection projection = basicProjection.copyOf() + .attribute(Attribute.builder() + .type(Date.class) + .name("recordedDate") + .argument(Argument.builder() + .name("grain") + .value("year") + .build()) + .build()) + .build(); + + assertThrows(InvalidOperationException.class, () -> new EntityProjectionTranslator( + playerStatsTable, + projection, + dictionary + )); + } +} diff --git a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/QueryValidatorTest.java b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/QueryValidatorTest.java new file mode 100644 index 0000000000..e0b7f2ba45 --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/QueryValidatorTest.java @@ -0,0 +1,204 @@ +/* + * Copyright 2020, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation; + +import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.junit.jupiter.api.Assertions.assertThrows; + +import com.yahoo.elide.core.exceptions.InvalidOperationException; +import com.yahoo.elide.core.filter.dialect.ParseException; +import com.yahoo.elide.core.filter.expression.FilterExpression; +import com.yahoo.elide.core.sort.Sorting; +import com.yahoo.elide.datastores.aggregation.example.PlayerStats; +import com.yahoo.elide.datastores.aggregation.filter.visitor.FilterConstraints; +import com.yahoo.elide.datastores.aggregation.filter.visitor.SplitFilterExpressionVisitor; +import com.yahoo.elide.datastores.aggregation.framework.SQLUnitTest; +import com.yahoo.elide.datastores.aggregation.query.Query; + +import org.junit.jupiter.api.BeforeAll; +import org.junit.jupiter.api.Test; + +import java.util.Arrays; +import java.util.Collections; +import java.util.HashSet; +import java.util.Map; +import java.util.Set; +import java.util.TreeMap; + +public class QueryValidatorTest extends SQLUnitTest { + @BeforeAll + public static void init() { + SQLUnitTest.init(); + } + + @Test + public void testNoMetricQuery() { + Map sortMap = new TreeMap<>(); + sortMap.put("country.name", Sorting.SortOrder.asc); + + Query query = Query.builder() + .analyticView(playerStatsTable) + .groupByDimension(toProjection(playerStatsTable.getDimension("overallRating"))) + .sorting(new Sorting(sortMap)) + .build(); + + QueryValidator validator = new QueryValidator(query, Collections.singleton("overallRating"), dictionary); + + UnsupportedOperationException exception = assertThrows(UnsupportedOperationException.class, validator::validate); + assertEquals("Query with no metric can't sort on nested field.", exception.getMessage()); + } + + @Test + public void testSortingOnId() { + Map sortMap = new TreeMap<>(); + sortMap.put("id", Sorting.SortOrder.asc); + + Query query = Query.builder() + .analyticView(playerStatsTable) + .metric(invoke(playerStatsTable.getMetric("lowScore"))) + .groupByDimension(toProjection(playerStatsTable.getDimension("id"))) + .groupByDimension(toProjection(playerStatsTable.getDimension("overallRating"))) + .sorting(new Sorting(sortMap)) + .build(); + + Set allFields = new HashSet<>(Arrays.asList("id", "overallRating", "lowScore")); + QueryValidator validator = new QueryValidator(query, allFields, dictionary); + + InvalidOperationException exception = assertThrows(InvalidOperationException.class, validator::validate); + assertEquals("Invalid operation: 'Sorting on id field is not permitted'", exception.getMessage()); + } + + @Test + public void testSortingOnNotQueriedDimension() { + Map sortMap = new TreeMap<>(); + sortMap.put("country.name", Sorting.SortOrder.asc); + + Query query = Query.builder() + .analyticView(playerStatsTable) + .metric(invoke(playerStatsTable.getMetric("lowScore"))) + .groupByDimension(toProjection(playerStatsTable.getDimension("overallRating"))) + .sorting(new Sorting(sortMap)) + .build(); + + Set allFields = new HashSet<>(Arrays.asList("overallRating", "lowScore")); + QueryValidator validator = new QueryValidator(query, allFields, dictionary); + + InvalidOperationException exception = assertThrows(InvalidOperationException.class, validator::validate); + assertEquals("Invalid operation: 'Can't sort on country as it is not present in query'", exception.getMessage()); + } + + @Test + public void testSortingOnNotQueriedMetric() { + Map sortMap = new TreeMap<>(); + sortMap.put("highScore", Sorting.SortOrder.asc); + + Query query = Query.builder() + .analyticView(playerStatsTable) + .metric(invoke(playerStatsTable.getMetric("lowScore"))) + .groupByDimension(toProjection(playerStatsTable.getDimension("overallRating"))) + .sorting(new Sorting(sortMap)) + .build(); + + Set allFields = new HashSet<>(Arrays.asList("overallRating", "lowScore")); + QueryValidator validator = new QueryValidator(query, allFields, dictionary); + + InvalidOperationException exception = assertThrows(InvalidOperationException.class, validator::validate); + assertEquals("Invalid operation: 'Can't sort on highScore as it is not present in query'", exception.getMessage()); + } + + @Test + public void testSortingOnNestedDimensionField() { + Map sortMap = new TreeMap<>(); + sortMap.put("country.continent.name", Sorting.SortOrder.asc); + + Query query = Query.builder() + .analyticView(playerStatsTable) + .metric(invoke(playerStatsTable.getMetric("lowScore"))) + .groupByDimension(toProjection(playerStatsTable.getDimension("country"))) + .sorting(new Sorting(sortMap)) + .build(); + + Set allFields = new HashSet<>(Arrays.asList("country", "lowScore")); + QueryValidator validator = new QueryValidator(query, allFields, dictionary); + + UnsupportedOperationException exception = assertThrows(UnsupportedOperationException.class, validator::validate); + assertEquals("Currently sorting on double nested fields is not supported", exception.getMessage()); + } + + @Test + public void testHavingFilterPromotionUngroupedDimension() throws ParseException { + FilterExpression originalFilter = filterParser.parseFilterExpression("countryIsoCode==USA,lowScore<45", + PlayerStats.class, false); + SplitFilterExpressionVisitor visitor = new SplitFilterExpressionVisitor(playerStatsTable); + FilterConstraints constraints = originalFilter.accept(visitor); + FilterExpression whereFilter = constraints.getWhereExpression(); + FilterExpression havingFilter = constraints.getHavingExpression(); + + Query query = Query.builder() + .analyticView(playerStatsTable) + .metric(invoke(playerStatsTable.getMetric("lowScore"))) + .whereFilter(whereFilter) + .havingFilter(havingFilter) + .build(); + + Set allFields = new HashSet<>(Collections.singletonList("lowScore")); + QueryValidator validator = new QueryValidator(query, allFields, dictionary); + + InvalidOperationException exception = assertThrows(InvalidOperationException.class, validator::validate); + assertEquals( + "Invalid operation: 'Dimension field countryIsoCode must be grouped before filtering in having clause.'", + exception.getMessage()); + } + + @Test + public void testHavingFilterNoAggregatedMetric() throws ParseException { + FilterExpression originalFilter = filterParser.parseFilterExpression("lowScore<45", PlayerStats.class, false); + SplitFilterExpressionVisitor visitor = new SplitFilterExpressionVisitor(playerStatsTable); + FilterConstraints constraints = originalFilter.accept(visitor); + FilterExpression whereFilter = constraints.getWhereExpression(); + FilterExpression havingFilter = constraints.getHavingExpression(); + + Query query = Query.builder() + .analyticView(playerStatsTable) + .metric(invoke(playerStatsTable.getMetric("highScore"))) + .whereFilter(whereFilter) + .havingFilter(havingFilter) + .build(); + + Set allFields = new HashSet<>(Collections.singletonList("highScore")); + QueryValidator validator = new QueryValidator(query, allFields, dictionary); + + InvalidOperationException exception = assertThrows(InvalidOperationException.class, validator::validate); + assertEquals( + "Invalid operation: 'Metric field lowScore must be aggregated before filtering in having clause.'", + exception.getMessage()); + } + + @Test + public void testHavingFilterOnDimensionTable() throws ParseException { + FilterExpression originalFilter = filterParser.parseFilterExpression("country.isoCode==USA,lowScore<45", + PlayerStats.class, false); + SplitFilterExpressionVisitor visitor = new SplitFilterExpressionVisitor(playerStatsTable); + FilterConstraints constraints = originalFilter.accept(visitor); + FilterExpression whereFilter = constraints.getWhereExpression(); + FilterExpression havingFilter = constraints.getHavingExpression(); + + Query query = Query.builder() + .analyticView(playerStatsTable) + .metric(invoke(playerStatsTable.getMetric("lowScore"))) + .whereFilter(whereFilter) + .havingFilter(havingFilter) + .build(); + + Set allFields = new HashSet<>(Collections.singletonList("lowScore")); + QueryValidator validator = new QueryValidator(query, allFields, dictionary); + + InvalidOperationException exception = assertThrows(InvalidOperationException.class, validator::validate); + assertEquals( + "Invalid operation: 'Can't filter on relationship field [PlayerStats].country/[Country].isoCode in HAVING clause when querying table PlayerStats.'", + exception.getMessage()); + } +} diff --git a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/Continent.java b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/Continent.java new file mode 100644 index 0000000000..0a557a9b2f --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/Continent.java @@ -0,0 +1,28 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.example; + +import com.yahoo.elide.annotation.Include; +import com.yahoo.elide.datastores.aggregation.annotation.Cardinality; +import com.yahoo.elide.datastores.aggregation.annotation.CardinalitySize; +import lombok.Data; + +import javax.persistence.Entity; +import javax.persistence.Id; +import javax.persistence.Table; + +@Data +@Entity +@Include(rootLevel = true) +@Table(name = "continents") +@Cardinality(size = CardinalitySize.SMALL) +public class Continent { + + @Id + private String id; + + private String name; +} diff --git a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/Country.java b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/Country.java new file mode 100644 index 0000000000..4e43e7b0e0 --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/Country.java @@ -0,0 +1,73 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.example; + +import com.yahoo.elide.annotation.Include; +import com.yahoo.elide.datastores.aggregation.annotation.Cardinality; +import com.yahoo.elide.datastores.aggregation.annotation.CardinalitySize; +import com.yahoo.elide.datastores.aggregation.annotation.FriendlyName; +import lombok.Data; + +import javax.persistence.Entity; +import javax.persistence.Id; +import javax.persistence.JoinColumn; +import javax.persistence.ManyToOne; +import javax.persistence.Table; + +/** + * A root level entity for testing AggregationDataStore. + */ +@Data +@Entity +@Include(rootLevel = true) +@Table(name = "countries") +@Cardinality(size = CardinalitySize.SMALL) +public class Country { + + private String id; + + private String isoCode; + + private String name; + + private Continent continent; + + @Id + public String getId() { + return id; + } + + public void setId(final String id) { + this.id = id; + } + + public String getIsoCode() { + return isoCode; + } + + public void setIsoCode(final String isoCode) { + this.isoCode = isoCode; + } + + @FriendlyName + public String getName() { + return name; + } + + public void setName(final String name) { + this.name = name; + } + + @ManyToOne + @JoinColumn(name = "continent_id") + public Continent getContinent() { + return continent; + } + + public void setContinent(Continent continent) { + this.continent = continent; + } +} diff --git a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/CountryView.java b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/CountryView.java new file mode 100644 index 0000000000..0f983ff1ef --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/CountryView.java @@ -0,0 +1,80 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.example; + +import com.yahoo.elide.annotation.Include; +import com.yahoo.elide.annotation.ToOne; +import com.yahoo.elide.datastores.aggregation.annotation.FriendlyName; +import com.yahoo.elide.datastores.aggregation.queryengines.sql.annotation.JoinTo; + +import lombok.Data; + +import javax.persistence.Column; +import javax.persistence.Table; + +/** + * A view version of table countries. + */ +@Data +@Include +@Table(name = "countries") +public class CountryView { + @Column(name = "id") + private String countryId; + + private String isoCode; + + private String name; + + private CountryViewNested nestedView; + + @ToOne + @JoinTo( + joinClause = "%from.id = %join.id" + ) + public CountryViewNested getNestedView() { + return nestedView; + } + + @JoinTo(path = "nestedView.isoCode") + private String nestedViewIsoCode; + + private Country nestedRelationship; + + @ToOne + @Column(name = "id") + public Country getNestedRelationship() { + return nestedRelationship; + } + + @JoinTo(path = "nestedRelationship.isoCode") + private String nestedRelationshipIsoCode; + + public String getCountryId() { + return countryId; + } + + public void setCountryId(final String countryId) { + this.countryId = countryId; + } + + public String getIsoCode() { + return isoCode; + } + + public void setIsoCode(final String isoCode) { + this.isoCode = isoCode; + } + + @FriendlyName + public String getName() { + return name; + } + + public void setName(final String name) { + this.name = name; + } +} diff --git a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/CountryViewNested.java b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/CountryViewNested.java new file mode 100644 index 0000000000..c77ef81a0f --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/CountryViewNested.java @@ -0,0 +1,54 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.example; + +import com.yahoo.elide.annotation.Include; +import com.yahoo.elide.datastores.aggregation.annotation.FriendlyName; + +import lombok.Data; + +import javax.persistence.Id; +import javax.persistence.Table; + +/** + * A nested view for testing. + */ +@Data +@Include +@Table(name = "countries") +public class CountryViewNested { + private String id; + + private String isoCode; + + private String name; + + @Id + public String getId() { + return id; + } + + public void setId(final String id) { + this.id = id; + } + + public String getIsoCode() { + return isoCode; + } + + public void setIsoCode(final String isoCode) { + this.isoCode = isoCode; + } + + @FriendlyName + public String getName() { + return name; + } + + public void setName(final String name) { + this.name = name; + } +} diff --git a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/Player.java b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/Player.java new file mode 100644 index 0000000000..c8b0c8c718 --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/Player.java @@ -0,0 +1,34 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.example; + +import com.yahoo.elide.annotation.Include; +import com.yahoo.elide.datastores.aggregation.annotation.Cardinality; +import com.yahoo.elide.datastores.aggregation.annotation.CardinalitySize; +import com.yahoo.elide.datastores.aggregation.annotation.FriendlyName; + +import lombok.Data; + +import javax.persistence.Entity; +import javax.persistence.Id; +import javax.persistence.Table; + +/** + * A root level entity for testing AggregationDataStore. + */ +@Entity +@Include(rootLevel = true) +@Table(name = "players") +@Cardinality(size = CardinalitySize.MEDIUM) +@Data +public class Player { + + @Id + private long id; + + @FriendlyName + private String name; +} diff --git a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/PlayerStats.java b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/PlayerStats.java new file mode 100644 index 0000000000..e5157c49fe --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/PlayerStats.java @@ -0,0 +1,235 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.example; + +import com.yahoo.elide.annotation.Include; +import com.yahoo.elide.datastores.aggregation.annotation.Cardinality; +import com.yahoo.elide.datastores.aggregation.annotation.CardinalitySize; +import com.yahoo.elide.datastores.aggregation.annotation.FriendlyName; +import com.yahoo.elide.datastores.aggregation.annotation.Meta; +import com.yahoo.elide.datastores.aggregation.annotation.MetricAggregation; +import com.yahoo.elide.datastores.aggregation.annotation.Temporal; +import com.yahoo.elide.datastores.aggregation.annotation.TimeGrainDefinition; +import com.yahoo.elide.datastores.aggregation.queryengines.sql.annotation.FromTable; +import com.yahoo.elide.datastores.aggregation.queryengines.sql.annotation.JoinTo; +import com.yahoo.elide.datastores.aggregation.queryengines.sql.metric.functions.SqlMax; +import com.yahoo.elide.datastores.aggregation.queryengines.sql.metric.functions.SqlMin; +import com.yahoo.elide.datastores.aggregation.time.TimeGrain; + +import lombok.EqualsAndHashCode; +import lombok.Setter; +import lombok.ToString; + +import java.util.Date; +import javax.persistence.Column; +import javax.persistence.Id; +import javax.persistence.JoinColumn; +import javax.persistence.ManyToOne; +/** + * A root level entity for testing AggregationDataStore. + */ +@Include(rootLevel = true) +@Cardinality(size = CardinalitySize.LARGE) +@EqualsAndHashCode +@ToString +@FromTable(name = "playerStats") +public class PlayerStats { + + public static final String DAY_FORMAT = "PARSEDATETIME(FORMATDATETIME(%s, 'yyyy-MM-dd'), 'yyyy-MM-dd')"; + public static final String MONTH_FORMAT = "PARSEDATETIME(FORMATDATETIME(%s, 'yyyy-MM-01'), 'yyyy-MM-dd')"; + + /** + * PK. + */ + private String id; + + /** + * A metric. + */ + private long highScore; + + /** + * A metric. + */ + private long lowScore; + + /** + * A degenerate dimension. + */ + private String overallRating; + + /** + * A table dimension. + */ + private Country country; + + /** + * A subselect dimension. + */ + private SubCountry subCountry; + + @Setter + private String countryViewIsoCode; + + /** + * A dimension field joined to this table. + */ + private String countryIsoCode; + + /** + * A dimension field joined to this table. + */ + private String subCountryIsoCode; + + /** + * A table dimension. + */ + private Player player; + + /** + * A table dimension. + */ + private Player player2; + + private String playerName; + + private String player2Name; + + private Date recordedDate; + + @Id + public String getId() { + return id; + } + + public void setId(final String id) { + this.id = id; + } + + @MetricAggregation(function = SqlMax.class) + @Meta(longName = "awesome score", description = "very awesome score") + public long getHighScore() { + return highScore; + } + + public void setHighScore(final long highScore) { + this.highScore = highScore; + } + + @MetricAggregation(function = SqlMin.class) + public long getLowScore() { + return lowScore; + } + + public void setLowScore(final long lowScore) { + this.lowScore = lowScore; + } + + @FriendlyName + @Cardinality(size = CardinalitySize.MEDIUM) + public String getOverallRating() { + return overallRating; + } + + public void setOverallRating(final String overallRating) { + this.overallRating = overallRating; + } + + @ManyToOne + @JoinColumn(name = "country_id") + public Country getCountry() { + return country; + } + + public void setCountry(final Country country) { + this.country = country; + } + + @ManyToOne + @JoinColumn(name = "sub_country_id") + public SubCountry getSubCountry() { + return subCountry; + } + + public void setSubCountry(final SubCountry subCountry) { + this.subCountry = subCountry; + } + + @ManyToOne + @JoinColumn(name = "player_id") + public Player getPlayer() { + return player; + } + + public void setPlayer(final Player player) { + this.player = player; + } + + /** + * DO NOT put {@link Cardinality} annotation on this field. See + * + * @return the date of the player session. + */ + @Temporal(grains = { + @TimeGrainDefinition(grain = TimeGrain.DAY, expression = DAY_FORMAT), + @TimeGrainDefinition(grain = TimeGrain.MONTH, expression = MONTH_FORMAT) + }, timeZone = "UTC") + public Date getRecordedDate() { + return recordedDate; + } + + public void setRecordedDate(final Date recordedDate) { + this.recordedDate = recordedDate; + } + + @JoinTo(path = "country.isoCode") + public String getCountryIsoCode() { + return countryIsoCode; + } + + public void setCountryIsoCode(String isoCode) { + this.countryIsoCode = isoCode; + } + + + @JoinTo(path = "subCountry.isoCode") + @Column(updatable = false, insertable = false) // subselect field should be read-only + public String getSubCountryIsoCode() { + return subCountryIsoCode; + } + + public void setSubCountryIsoCode(String isoCode) { + this.subCountryIsoCode = isoCode; + } + + @JoinColumn(name = "player2_id") + @ManyToOne + public Player getPlayer2() { + return player2; + } + + public void setPlayer2(Player player2) { + this.player2 = player2; + } + + @JoinTo(path = "player.name") + public String getPlayerName() { + return playerName; + } + + public void setPlayerName(String playerName) { + this.playerName = playerName; + } + + @JoinTo(path = "player2.name") + public String getPlayer2Name() { + return player2Name; + } + + public void setPlayer2Name(String player2Name) { + this.player2Name = player2Name; + } +} diff --git a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/PlayerStatsView.java b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/PlayerStatsView.java new file mode 100644 index 0000000000..22562b204a --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/PlayerStatsView.java @@ -0,0 +1,46 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.example; + +import com.yahoo.elide.annotation.Include; +import com.yahoo.elide.datastores.aggregation.annotation.MetricAggregation; +import com.yahoo.elide.datastores.aggregation.queryengines.sql.annotation.FromSubquery; +import com.yahoo.elide.datastores.aggregation.queryengines.sql.metric.functions.SqlMax; +import lombok.Data; + +import javax.persistence.Id; +import javax.persistence.JoinColumn; +import javax.persistence.OneToOne; + +/** + * A root level entity for testing AggregationDataStore. + */ +@Include(rootLevel = true) +@Data +@FromSubquery(sql = "SELECT stats.highScore, stats.player_id, c.name as countryName FROM playerStats AS stats LEFT JOIN countries AS c ON stats.country_id = c.id WHERE stats.overallRating = 'Great'") +public class PlayerStatsView { + + /** + * PK. + */ + @Id + private String id; + + /** + * A metric. + */ + @MetricAggregation(function = SqlMax.class) + private long highScore; + + /** + * A degenerate dimension. + */ + private String countryName; + + @OneToOne + @JoinColumn(name = "player_id") + private Player player; +} diff --git a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/PlayerStatsWithView.java b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/PlayerStatsWithView.java new file mode 100644 index 0000000000..7541c38775 --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/PlayerStatsWithView.java @@ -0,0 +1,223 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.example; + +import com.yahoo.elide.annotation.Include; +import com.yahoo.elide.annotation.ToOne; +import com.yahoo.elide.datastores.aggregation.annotation.Cardinality; +import com.yahoo.elide.datastores.aggregation.annotation.CardinalitySize; +import com.yahoo.elide.datastores.aggregation.annotation.FriendlyName; +import com.yahoo.elide.datastores.aggregation.annotation.Meta; +import com.yahoo.elide.datastores.aggregation.annotation.MetricAggregation; +import com.yahoo.elide.datastores.aggregation.annotation.Temporal; +import com.yahoo.elide.datastores.aggregation.annotation.TimeGrainDefinition; +import com.yahoo.elide.datastores.aggregation.queryengines.sql.annotation.FromTable; +import com.yahoo.elide.datastores.aggregation.queryengines.sql.annotation.JoinTo; +import com.yahoo.elide.datastores.aggregation.queryengines.sql.metric.functions.SqlMax; +import com.yahoo.elide.datastores.aggregation.queryengines.sql.metric.functions.SqlMin; +import com.yahoo.elide.datastores.aggregation.time.TimeGrain; + +import lombok.EqualsAndHashCode; +import lombok.Setter; +import lombok.ToString; + +import java.util.Date; +import javax.persistence.Column; +import javax.persistence.Id; +import javax.persistence.JoinColumn; +import javax.persistence.ManyToOne; + +/** + * A root level entity for testing AggregationDataStore. + */ +@Include(rootLevel = true) +@Cardinality(size = CardinalitySize.LARGE) +@EqualsAndHashCode +@ToString +@FromTable(name = "playerStats") +public class PlayerStatsWithView { + + /** + * PK. + */ + private String id; + + /** + * A metric. + */ + private long highScore; + + /** + * A metric. + */ + private long lowScore; + + /** + * A degenerate dimension. + */ + private String overallRating; + + /** + * A table dimension. + */ + private Country country; + + /** + * A subselect dimension. + */ + private SubCountry subCountry; + + private CountryView countryView; + + @Setter + private String countryViewIsoCode; + + @Setter + private String countryViewViewIsoCode; + + @Setter + private String countryViewRelationshipIsoCode; + + /** + * A dimension field joined to this table. + */ + private String countryIsoCode; + + /** + * A dimension field joined to this table. + */ + private String subCountryIsoCode; + + /** + * A table dimension. + */ + private Player player; + + private Date recordedDate; + + @Id + public String getId() { + return id; + } + + public void setId(final String id) { + this.id = id; + } + + @MetricAggregation(function = SqlMax.class) + @Meta(longName = "awesome score", description = "very awesome score") + public long getHighScore() { + return highScore; + } + + public void setHighScore(final long highScore) { + this.highScore = highScore; + } + + @MetricAggregation(function = SqlMin.class) + public long getLowScore() { + return lowScore; + } + + public void setLowScore(final long lowScore) { + this.lowScore = lowScore; + } + + @FriendlyName + @Cardinality(size = CardinalitySize.MEDIUM) + public String getOverallRating() { + return overallRating; + } + + public void setOverallRating(final String overallRating) { + this.overallRating = overallRating; + } + + @ManyToOne + @JoinColumn(name = "country_id") + public Country getCountry() { + return country; + } + + public void setCountry(final Country country) { + this.country = country; + } + + @ManyToOne + @JoinColumn(name = "sub_country_id") + public SubCountry getSubCountry() { + return subCountry; + } + + public void setSubCountry(final SubCountry subCountry) { + this.subCountry = subCountry; + } + + @ManyToOne + @JoinColumn(name = "player_id") + public Player getPlayer() { + return player; + } + + public void setPlayer(final Player player) { + this.player = player; + } + + /** + * DO NOT put {@link Cardinality} annotation on this field. + * + * @return the date of the player session. + */ + @Temporal(grains = { @TimeGrainDefinition(grain = TimeGrain.DAY, expression = "") }, timeZone = "UTC") + public Date getRecordedDate() { + return recordedDate; + } + + public void setRecordedDate(final Date recordedDate) { + this.recordedDate = recordedDate; + } + + @JoinTo(path = "country.isoCode") + public String getCountryIsoCode() { + return countryIsoCode; + } + + public void setCountryIsoCode(String isoCode) { + this.countryIsoCode = isoCode; + } + + + @JoinTo(path = "subCountry.isoCode") + @Column(updatable = false, insertable = false) // subselect field should be read-only + public String getSubCountryIsoCode() { + return subCountryIsoCode; + } + + public void setSubCountryIsoCode(String isoCode) { + this.subCountryIsoCode = isoCode; + } + + @ToOne + @JoinTo(joinClause = "%from.country_id = %join.id") + public CountryView getCountryView() { + return countryView; + } + + @JoinTo(path = "countryView.isoCode") + public String getCountryViewIsoCode() { + return countryViewIsoCode; + } + + @JoinTo(path = "countryView.nestedView.isoCode") + public String getCountryViewViewIsoCode() { + return countryViewViewIsoCode; + } + + @JoinTo(path = "countryView.nestedRelationship.isoCode") + public String getCountryViewRelationshipIsoCode() { + return countryViewRelationshipIsoCode; + } +} diff --git a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/SubCountry.java b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/SubCountry.java new file mode 100644 index 0000000000..f6c14c611e --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/SubCountry.java @@ -0,0 +1,61 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.example; + +import com.yahoo.elide.annotation.Include; +import com.yahoo.elide.datastores.aggregation.annotation.Cardinality; +import com.yahoo.elide.datastores.aggregation.annotation.CardinalitySize; +import com.yahoo.elide.datastores.aggregation.annotation.FriendlyName; + +import org.hibernate.annotations.Subselect; + +import lombok.Data; + +import javax.persistence.Entity; +import javax.persistence.Id; + +/** + * A root level entity for testing AggregationDataStore with @Subselect annotation. + */ +@Data +@Entity +@Include(rootLevel = true) +@Subselect(value = "select * from countries") +@Cardinality(size = CardinalitySize.SMALL) +public class SubCountry { + + private String id; + + private String isoCode; + + private String name; + + @Id + public String getId() { + return id; + } + + public void setId(final String id) { + this.id = id; + } + + public String getIsoCode() { + return isoCode; + } + + public void setIsoCode(final String isoCode) { + this.isoCode = isoCode; + } + + @FriendlyName + public String getName() { + return name; + } + + public void setName(final String name) { + this.name = name; + } +} diff --git a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/VideoGame.java b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/VideoGame.java new file mode 100644 index 0000000000..fa45f179d8 --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/VideoGame.java @@ -0,0 +1,79 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.example; + +import com.yahoo.elide.annotation.Include; +import com.yahoo.elide.datastores.aggregation.annotation.MetricAggregation; +import com.yahoo.elide.datastores.aggregation.annotation.MetricComputation; +import com.yahoo.elide.datastores.aggregation.queryengines.sql.annotation.FromTable; +import com.yahoo.elide.datastores.aggregation.queryengines.sql.metric.functions.SqlSum; + +import javax.persistence.Column; +import javax.persistence.Id; + +/** + * A root level entity for testing AggregationDataStore. + */ +@Include(rootLevel = true) +@FromTable(name = "videoGames") +public class VideoGame { + + @Id + private Long id; + + @Column(name = "game_rounds") + @MetricAggregation(function = SqlSum.class) + Long sessions; + + @MetricAggregation(function = SqlSum.class) + Long timeSpent; + + @MetricComputation(expression = "timeSpent / sessions") + private Float timeSpentPerSession; + + @MetricComputation(expression = "timeSpentPerSession / 100") + private Float timeSpentPerGame; + + public Long getId() { + return id; + } + + public void setId(final Long id) { + this.id = id; + } + + public Long getSessions() { + return sessions; + } + + public void setSessions(final Long sessions) { + this.sessions = sessions; + } + + public Long getTimeSpent() { + return timeSpent; + } + + public void setTimeSpent(final Long timeSpent) { + this.timeSpent = timeSpent; + } + + public Float getTimeSpentPerSession() { + return timeSpentPerSession; + } + + public void setTimeSpentPerSession(final Float timeSpentPerSession) { + this.timeSpentPerSession = timeSpentPerSession; + } + + public Float getTimeSpentPerGame() { + return timeSpentPerGame; + } + + public void setTimeSpentPerGame(final Float timeSpentPerGame) { + this.timeSpentPerGame = timeSpentPerGame; + } +} diff --git a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/filter/visitor/FilterConstraintsTest.java b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/filter/visitor/FilterConstraintsTest.java new file mode 100644 index 0000000000..2ee7bad495 --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/filter/visitor/FilterConstraintsTest.java @@ -0,0 +1,70 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.filter.visitor; + +import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.junit.jupiter.api.Assertions.assertFalse; +import static org.junit.jupiter.api.Assertions.assertNull; +import static org.junit.jupiter.api.Assertions.assertTrue; + +import com.yahoo.elide.core.Path; +import com.yahoo.elide.core.filter.FilterPredicate; +import com.yahoo.elide.core.filter.Operator; +import com.yahoo.elide.datastores.aggregation.example.PlayerStats; + +import org.junit.jupiter.api.Test; + +import java.util.Collections; + +public class FilterConstraintsTest { + + private static final FilterPredicate WHERE_PREDICATE = new FilterPredicate( + new Path.PathElement(PlayerStats.class, String.class, "id"), + Operator.IN, + Collections.singletonList("foo") + ); + private static final FilterPredicate HAVING_PREDICATE = new FilterPredicate( + new Path.PathElement(PlayerStats.class, long.class, "highScore"), + Operator.GT, + Collections.singletonList(99) + ); + + @Test + public void testPureHaving() { + assertTrue(FilterConstraints.pureHaving(HAVING_PREDICATE).isPureHaving()); + assertFalse(FilterConstraints.pureHaving(HAVING_PREDICATE).isPureWhere()); + assertEquals( + "playerStats.highScore GT [99]", + FilterConstraints.pureHaving(HAVING_PREDICATE).getHavingExpression().toString() + ); + assertNull(FilterConstraints.pureHaving(HAVING_PREDICATE).getWhereExpression()); + } + + @Test + public void testPureWhere() { + assertTrue(FilterConstraints.pureWhere(WHERE_PREDICATE).isPureWhere()); + assertFalse(FilterConstraints.pureWhere(WHERE_PREDICATE).isPureHaving()); + assertEquals( + "playerStats.id IN [foo]", + FilterConstraints.pureWhere(WHERE_PREDICATE).getWhereExpression().toString() + ); + assertNull(FilterConstraints.pureWhere(WHERE_PREDICATE).getHavingExpression()); + } + + @Test + public void testWithWhereAndHaving() { + assertFalse(FilterConstraints.withWhereAndHaving(WHERE_PREDICATE, HAVING_PREDICATE).isPureWhere()); + assertFalse(FilterConstraints.withWhereAndHaving(WHERE_PREDICATE, HAVING_PREDICATE).isPureHaving()); + assertEquals( + "playerStats.id IN [foo]", + FilterConstraints.withWhereAndHaving(WHERE_PREDICATE, HAVING_PREDICATE).getWhereExpression().toString() + ); + assertEquals( + "playerStats.highScore GT [99]", + FilterConstraints.withWhereAndHaving(WHERE_PREDICATE, HAVING_PREDICATE).getHavingExpression().toString() + ); + } +} diff --git a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/filter/visitor/SplitFilterExpressionVisitorTest.java b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/filter/visitor/SplitFilterExpressionVisitorTest.java new file mode 100644 index 0000000000..50ce0dbf4d --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/filter/visitor/SplitFilterExpressionVisitorTest.java @@ -0,0 +1,176 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.filter.visitor; + +import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.junit.jupiter.api.Assertions.assertFalse; +import static org.junit.jupiter.api.Assertions.assertNull; +import static org.junit.jupiter.api.Assertions.assertTrue; + +import com.yahoo.elide.core.EntityDictionary; +import com.yahoo.elide.core.Path; +import com.yahoo.elide.core.filter.FilterPredicate; +import com.yahoo.elide.core.filter.Operator; +import com.yahoo.elide.core.filter.expression.AndFilterExpression; +import com.yahoo.elide.core.filter.expression.FilterExpressionVisitor; +import com.yahoo.elide.core.filter.expression.NotFilterExpression; +import com.yahoo.elide.core.filter.expression.OrFilterExpression; +import com.yahoo.elide.datastores.aggregation.example.Country; +import com.yahoo.elide.datastores.aggregation.example.Player; +import com.yahoo.elide.datastores.aggregation.example.PlayerStats; +import com.yahoo.elide.datastores.aggregation.example.SubCountry; +import com.yahoo.elide.datastores.aggregation.metadata.models.Table; + +import org.junit.jupiter.api.BeforeAll; +import org.junit.jupiter.api.Test; + +import java.util.Collections; + +public class SplitFilterExpressionVisitorTest { + + private static final FilterPredicate WHERE_PREDICATE = new FilterPredicate( + new Path.PathElement(PlayerStats.class, String.class, "id"), + Operator.IN, + Collections.singletonList("foo") + ); + private static final FilterPredicate HAVING_PREDICATE = new FilterPredicate( + new Path.PathElement(PlayerStats.class, long.class, "highScore"), + Operator.GT, + Collections.singletonList(99) + ); + + private static FilterExpressionVisitor splitFilterExpressionVisitor; + + @BeforeAll + public static void setupEntityDictionary() { + EntityDictionary entityDictionary = new EntityDictionary(Collections.emptyMap()); + entityDictionary.bindEntity(PlayerStats.class); + entityDictionary.bindEntity(Country.class); + entityDictionary.bindEntity(SubCountry.class); + entityDictionary.bindEntity(Player.class); + Table table = new Table(PlayerStats.class, entityDictionary); + splitFilterExpressionVisitor = new SplitFilterExpressionVisitor(table); + } + + @Test + public void testVisitPredicate() { + // predicate should be a WHERE + assertTrue(splitFilterExpressionVisitor.visitPredicate(WHERE_PREDICATE).isPureWhere()); + assertEquals( + "playerStats.id IN [foo]", + splitFilterExpressionVisitor.visitPredicate(WHERE_PREDICATE).getWhereExpression().toString() + ); + assertFalse(splitFilterExpressionVisitor.visitPredicate(WHERE_PREDICATE).isPureHaving()); + assertNull(splitFilterExpressionVisitor.visitPredicate(WHERE_PREDICATE).getHavingExpression()); + + // predicate should be a HAVING + assertTrue(splitFilterExpressionVisitor.visitPredicate(HAVING_PREDICATE).isPureHaving()); + assertEquals( + "playerStats.highScore GT [99]", + splitFilterExpressionVisitor.visitPredicate(HAVING_PREDICATE).getHavingExpression().toString() + ); + assertFalse(splitFilterExpressionVisitor.visitPredicate(HAVING_PREDICATE).isPureWhere()); + assertNull(splitFilterExpressionVisitor.visitPredicate(HAVING_PREDICATE).getWhereExpression()); + } + + @Test + public void testVisitAndExpression() { + // pure-W AND pure-W + AndFilterExpression filterExpression = new AndFilterExpression(WHERE_PREDICATE, WHERE_PREDICATE); + assertEquals( + "(playerStats.id IN [foo] AND playerStats.id IN [foo])", + splitFilterExpressionVisitor.visitAndExpression(filterExpression).getWhereExpression().toString() + ); + assertNull(splitFilterExpressionVisitor.visitAndExpression(filterExpression).getHavingExpression()); + + // pure-H AND pure-W + filterExpression = new AndFilterExpression(HAVING_PREDICATE, WHERE_PREDICATE); + assertEquals( + "playerStats.id IN [foo]", + splitFilterExpressionVisitor.visitAndExpression(filterExpression).getWhereExpression().toString() + ); + assertEquals( + "playerStats.highScore GT [99]", + splitFilterExpressionVisitor.visitAndExpression(filterExpression).getHavingExpression().toString() + ); + + // pure-W AND pure-H + filterExpression = new AndFilterExpression(WHERE_PREDICATE, HAVING_PREDICATE); + assertEquals( + "playerStats.id IN [foo]", + splitFilterExpressionVisitor.visitAndExpression(filterExpression).getWhereExpression().toString() + ); + assertEquals( + "playerStats.highScore GT [99]", + splitFilterExpressionVisitor.visitAndExpression(filterExpression).getHavingExpression().toString() + ); + + // non-pure case - H1 AND W1 AND H2 + AndFilterExpression and1 = new AndFilterExpression(HAVING_PREDICATE, WHERE_PREDICATE); + AndFilterExpression and2 = new AndFilterExpression(and1, HAVING_PREDICATE); + assertEquals( + "playerStats.id IN [foo]", + splitFilterExpressionVisitor.visitAndExpression(and2).getWhereExpression().toString() + ); + assertEquals( + "(playerStats.highScore GT [99] AND playerStats.highScore GT [99])", + splitFilterExpressionVisitor.visitAndExpression(and2).getHavingExpression().toString() + ); + + // non-pure case - (H1 OR H2) AND W1 + OrFilterExpression or = new OrFilterExpression(HAVING_PREDICATE, HAVING_PREDICATE); + AndFilterExpression and = new AndFilterExpression(or, WHERE_PREDICATE); + assertEquals( + "playerStats.id IN [foo]", + splitFilterExpressionVisitor.visitAndExpression(and).getWhereExpression().toString() + ); + assertEquals( + "(playerStats.highScore GT [99] OR playerStats.highScore GT [99])", + splitFilterExpressionVisitor.visitAndExpression(and).getHavingExpression().toString() + ); + } + + @Test + public void testVisitOrExpression() { + // pure-W OR pure-W + OrFilterExpression filterExpression = new OrFilterExpression(WHERE_PREDICATE, WHERE_PREDICATE); + assertEquals( + "(playerStats.id IN [foo] OR playerStats.id IN [foo])", + splitFilterExpressionVisitor.visitOrExpression(filterExpression).getWhereExpression().toString() + ); + assertNull(splitFilterExpressionVisitor.visitOrExpression(filterExpression).getHavingExpression()); + + // H1 OR W1 + OrFilterExpression or = new OrFilterExpression(HAVING_PREDICATE, WHERE_PREDICATE); + assertNull(splitFilterExpressionVisitor.visitOrExpression(or).getWhereExpression()); + assertEquals( + "(playerStats.highScore GT [99] OR playerStats.id IN [foo])", + splitFilterExpressionVisitor.visitOrExpression(or).getHavingExpression().toString() + ); + + // (W1 AND H1) OR W2 + AndFilterExpression and = new AndFilterExpression(WHERE_PREDICATE, HAVING_PREDICATE); + or = new OrFilterExpression(and, WHERE_PREDICATE); + assertNull(splitFilterExpressionVisitor.visitOrExpression(or).getWhereExpression()); + assertEquals( + "((playerStats.id IN [foo] AND playerStats.highScore GT [99]) OR playerStats.id IN [foo])", + splitFilterExpressionVisitor.visitOrExpression(or).getHavingExpression().toString() + ); + } + + @Test + public void testVisitNotExpression() { + NotFilterExpression notExpression = new NotFilterExpression( + new AndFilterExpression(WHERE_PREDICATE, HAVING_PREDICATE) + ); + assertNull(splitFilterExpressionVisitor.visitNotExpression(notExpression).getWhereExpression()); + assertEquals( + "(playerStats.id NOT [foo] OR playerStats.highScore LE [99])", + splitFilterExpressionVisitor.visitNotExpression(notExpression).getHavingExpression().toString() + ); + + } +} diff --git a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/framework/AggregationDataStoreTestHarness.java b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/framework/AggregationDataStoreTestHarness.java new file mode 100644 index 0000000000..486dde0269 --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/framework/AggregationDataStoreTestHarness.java @@ -0,0 +1,42 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.framework; + +import com.yahoo.elide.core.DataStore; +import com.yahoo.elide.core.datastore.test.DataStoreTestHarness; +import com.yahoo.elide.datastores.aggregation.AggregationDataStore; +import com.yahoo.elide.datastores.aggregation.metadata.MetaDataStore; +import com.yahoo.elide.datastores.aggregation.queryengines.sql.SQLQueryEngineFactory; +import com.yahoo.elide.datastores.jpa.JpaDataStore; +import com.yahoo.elide.datastores.jpa.transaction.NonJtaTransaction; +import com.yahoo.elide.datastores.multiplex.MultiplexManager; + +public class AggregationDataStoreTestHarness implements DataStoreTestHarness { + private SQLQueryEngineFactory queryEngineFactory; + + public AggregationDataStoreTestHarness(SQLQueryEngineFactory queryEngineFactory) { + this.queryEngineFactory = queryEngineFactory; + } + + @Override + public DataStore getDataStore() { + MetaDataStore metaDataStore = new MetaDataStore(); + + AggregationDataStore aggregationDataStore = new AggregationDataStore(queryEngineFactory, metaDataStore); + + DataStore jpaStore = new JpaDataStore( + () -> queryEngineFactory.getEmf().createEntityManager(), + NonJtaTransaction::new + ); + + // meta data store needs to be put at first to populate meta data models + return new MultiplexManager(jpaStore, metaDataStore, aggregationDataStore); + } + + public void cleanseTestData() { + + } +} diff --git a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/framework/SQLUnitTest.java b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/framework/SQLUnitTest.java new file mode 100644 index 0000000000..0ad213230b --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/framework/SQLUnitTest.java @@ -0,0 +1,99 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.framework; + +import com.yahoo.elide.core.EntityDictionary; +import com.yahoo.elide.core.filter.dialect.RSQLFilterDialect; +import com.yahoo.elide.datastores.aggregation.QueryEngine; +import com.yahoo.elide.datastores.aggregation.example.Continent; +import com.yahoo.elide.datastores.aggregation.example.Country; +import com.yahoo.elide.datastores.aggregation.example.CountryView; +import com.yahoo.elide.datastores.aggregation.example.CountryViewNested; +import com.yahoo.elide.datastores.aggregation.example.Player; +import com.yahoo.elide.datastores.aggregation.example.PlayerStats; +import com.yahoo.elide.datastores.aggregation.example.PlayerStatsView; +import com.yahoo.elide.datastores.aggregation.example.PlayerStatsWithView; +import com.yahoo.elide.datastores.aggregation.example.SubCountry; +import com.yahoo.elide.datastores.aggregation.metadata.MetaDataStore; +import com.yahoo.elide.datastores.aggregation.metadata.metric.MetricFunctionInvocation; +import com.yahoo.elide.datastores.aggregation.metadata.models.AnalyticView; +import com.yahoo.elide.datastores.aggregation.metadata.models.Dimension; +import com.yahoo.elide.datastores.aggregation.metadata.models.Metric; +import com.yahoo.elide.datastores.aggregation.metadata.models.TimeDimension; +import com.yahoo.elide.datastores.aggregation.query.ColumnProjection; +import com.yahoo.elide.datastores.aggregation.query.TimeDimensionProjection; +import com.yahoo.elide.datastores.aggregation.queryengines.sql.SQLQueryEngine; +import com.yahoo.elide.datastores.aggregation.queryengines.sql.metadata.SQLAnalyticView; +import com.yahoo.elide.datastores.aggregation.time.TimeGrain; + +import java.util.Collections; +import java.util.HashMap; +import javax.persistence.EntityManagerFactory; +import javax.persistence.Persistence; + +public abstract class SQLUnitTest { + protected static EntityManagerFactory emf; + protected static AnalyticView playerStatsTable; + protected static EntityDictionary dictionary; + protected static RSQLFilterDialect filterParser; + protected static MetaDataStore metaDataStore = new MetaDataStore(); + + protected static final Country HONG_KONG = new Country(); + protected static final Country USA = new Country(); + protected static final Continent ASIA = new Continent(); + protected static final Continent NA = new Continent(); + + protected static QueryEngine engine; + + public static void init() { + emf = Persistence.createEntityManagerFactory("aggregationStore"); + dictionary = new EntityDictionary(new HashMap<>()); + dictionary.bindEntity(PlayerStatsWithView.class); + dictionary.bindEntity(PlayerStatsView.class); + dictionary.bindEntity(PlayerStats.class); + dictionary.bindEntity(Country.class); + dictionary.bindEntity(SubCountry.class); + dictionary.bindEntity(Player.class); + dictionary.bindEntity(CountryView.class); + dictionary.bindEntity(CountryViewNested.class); + dictionary.bindEntity(Continent.class); + filterParser = new RSQLFilterDialect(dictionary); + + playerStatsTable = new SQLAnalyticView(PlayerStats.class, dictionary); + + metaDataStore.populateEntityDictionary(dictionary); + + engine = new SQLQueryEngine(emf, metaDataStore); + + ASIA.setName("Asia"); + ASIA.setId("1"); + + NA.setName("North America"); + NA.setId("2"); + + HONG_KONG.setIsoCode("HKG"); + HONG_KONG.setName("Hong Kong"); + HONG_KONG.setId("344"); + HONG_KONG.setContinent(ASIA); + + USA.setIsoCode("USA"); + USA.setName("United States"); + USA.setId("840"); + USA.setContinent(NA); + } + + public static ColumnProjection toProjection(Dimension dimension) { + return ColumnProjection.toProjection(dimension, dimension.getName()); + } + + public static TimeDimensionProjection toProjection(TimeDimension dimension, TimeGrain grain) { + return ColumnProjection.toProjection(dimension, grain, dimension.getName()); + } + + public static MetricFunctionInvocation invoke(Metric metric) { + return metric.getMetricFunction().invoke(Collections.emptySet(), metric.getName()); + } +} diff --git a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/integration/AggregationDataStoreIntegrationTest.java b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/integration/AggregationDataStoreIntegrationTest.java new file mode 100644 index 0000000000..aa439d571e --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/integration/AggregationDataStoreIntegrationTest.java @@ -0,0 +1,947 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.integration; + +import static com.yahoo.elide.contrib.testhelpers.graphql.GraphQLDSL.argument; +import static com.yahoo.elide.contrib.testhelpers.graphql.GraphQLDSL.arguments; +import static com.yahoo.elide.contrib.testhelpers.graphql.GraphQLDSL.document; +import static com.yahoo.elide.contrib.testhelpers.graphql.GraphQLDSL.field; +import static com.yahoo.elide.contrib.testhelpers.graphql.GraphQLDSL.selection; +import static com.yahoo.elide.contrib.testhelpers.graphql.GraphQLDSL.selections; +import static io.restassured.RestAssured.given; +import static org.hamcrest.CoreMatchers.equalTo; +import static org.hamcrest.CoreMatchers.hasItem; +import static org.hamcrest.CoreMatchers.hasItems; +import static org.junit.jupiter.api.Assertions.assertEquals; + +import com.yahoo.elide.core.HttpStatus; +import com.yahoo.elide.core.datastore.test.DataStoreTestHarness; +import com.yahoo.elide.datastores.aggregation.AggregationDataStore; +import com.yahoo.elide.datastores.aggregation.framework.AggregationDataStoreTestHarness; +import com.yahoo.elide.datastores.aggregation.queryengines.sql.SQLQueryEngineFactory; +import com.yahoo.elide.initialization.IntegrationTest; + +import com.fasterxml.jackson.databind.JsonNode; +import com.fasterxml.jackson.databind.ObjectMapper; +import com.fasterxml.jackson.databind.node.ArrayNode; +import com.fasterxml.jackson.databind.node.JsonNodeFactory; +import com.fasterxml.jackson.databind.node.ObjectNode; +import org.junit.jupiter.api.Disabled; +import org.junit.jupiter.api.Test; + +import io.restassured.response.ValidatableResponse; + +import java.io.IOException; +import java.util.Map; +import javax.persistence.EntityManagerFactory; +import javax.persistence.Persistence; +import javax.ws.rs.core.MediaType; + +/** + * Integration tests for {@link AggregationDataStore}. + */ +public class AggregationDataStoreIntegrationTest extends IntegrationTest { + SQLQueryEngineFactory queryEngineFactory; + + private static final ObjectMapper JSON_MAPPER = new ObjectMapper(); + + @Override + protected DataStoreTestHarness createHarness() { + EntityManagerFactory emf = Persistence.createEntityManagerFactory("aggregationStore"); + queryEngineFactory = new SQLQueryEngineFactory(emf); + return new AggregationDataStoreTestHarness(queryEngineFactory); + } + + @Test + public void basicAggregationTest() throws Exception { + String graphQLRequest = document( + selection( + field( + "playerStats", + selections( + field("highScore"), + field("overallRating"), + field( + "country", + selections( + field("name") + ) + ) + ) + ) + ) + ).toQuery(); + + String expected = document( + selections( + field( + "playerStats", + selections( + field("highScore", 1234), + field("overallRating", "Good"), + field( + "country", + selections( + field("name", "United States") + ) + ) + ), + selections( + field("highScore", 2412), + field("overallRating", "Great"), + field( + "country", + selections( + field("name", "United States") + ) + ) + ), + selections( + field("highScore", 1000), + field("overallRating", "Good"), + field( + "country", + selections( + field("name", "Hong Kong") + ) + ) + ) + ) + ) + ).toResponse(); + + runQueryWithExpectedResult(graphQLRequest, expected); + } + + @Test + public void noMetricQueryTest() throws Exception { + String graphQLRequest = document( + selection( + field( + "playerStatsWithView", + arguments( + argument("sort", "\"countryViewRelationshipIsoCode\"") + ), + selections( + field( + "country", + selections( + field("name"), + field("isoCode") + ) + ), + field("countryViewRelationshipIsoCode") + ) + ) + ) + ).toQuery(); + + String expected = document( + selections( + field( + "playerStatsWithView", + selections( + field( + "country", + selections( + field("name", "Hong Kong"), + field("isoCode", "HKG") + ) + ), + field("countryViewRelationshipIsoCode", "HKG") + ), + selections( + field( + "country", + selections( + field("name", "United States"), + field("isoCode", "USA") + ) + ), + field("countryViewRelationshipIsoCode", "USA") + ) + ) + ) + ).toResponse(); + + runQueryWithExpectedResult(graphQLRequest, expected); + } + + @Test + public void whereFilterTest() throws Exception { + String graphQLRequest = document( + selection( + field( + "playerStats", + arguments( + argument("filter", "\"overallRating==\\\"Good\\\"\"") + ), + selections( + field("highScore"), + field("overallRating") + ) + ) + ) + ).toQuery(); + + String expected = document( + selections( + field( + "playerStats", + selections( + field("highScore", 1234), + field("overallRating", "Good") + ) + ) + ) + ).toResponse(); + + runQueryWithExpectedResult(graphQLRequest, expected); + } + + @Test + public void havingFilterTest() throws Exception { + String graphQLRequest = document( + selection( + field( + "playerStats", + arguments( + argument("filter", "\"lowScore<\\\"45\\\"\"") + ), + selections( + field("lowScore"), + field("overallRating"), + field( + "player", + selections( + field("name") + ) + ) + ) + ) + ) + ).toQuery(); + + String expected = document( + selections( + field( + "playerStats", + selections( + field("lowScore", 35), + field("overallRating", "Good"), + field( + "player", + selections( + field("name", "Jon Doe") + ) + ) + ) + ) + ) + ).toResponse(); + + runQueryWithExpectedResult(graphQLRequest, expected); + } + + /** + * Test the case that a where clause is promoted into having clause. + * @throws Exception exception + */ + @Test + public void wherePromotionTest() throws Exception { + String graphQLRequest = document( + selection( + field( + "playerStats", + arguments( + argument("filter", "\"overallRating==\\\"Good\\\",lowScore<\\\"45\\\"\"") + ), + selections( + field("lowScore"), + field("overallRating"), + field( + "player", + selections( + field("name") + ) + ) + ) + ) + ) + ).toQuery(); + + String expected = document( + selections( + field( + "playerStats", + selections( + field("lowScore", 35), + field("overallRating", "Good"), + field( + "player", + selections( + field("name", "Jon Doe") + ) + ) + ), + selections( + field("lowScore", 72), + field("overallRating", "Good"), + field( + "player", + selections( + field("name", "Han") + ) + ) + ) + ) + ) + ).toResponse(); + + runQueryWithExpectedResult(graphQLRequest, expected); + } + + /** + * Test the case that a where clause, which requires dimension join, is promoted into having clause. + * @throws Exception exception + */ + @Test + public void havingClauseJoinTest() throws Exception { + String graphQLRequest = document( + selection( + field( + "playerStats", + arguments( + argument("filter", "\"countryIsoCode==\\\"USA\\\",lowScore<\\\"45\\\"\""), + argument("sort", "\"lowScore\"") + ), + selections( + field("lowScore"), + field("countryIsoCode"), + field( + "player", + selections( + field("name") + ) + ) + ) + ) + ) + ).toQuery(); + + String expected = document( + selections( + field( + "playerStats", + selections( + field("lowScore", 35), + field("countryIsoCode", "USA"), + field( + "player", + selections( + field("name", "Jon Doe") + ) + ) + ), + selections( + field("lowScore", 241), + field("countryIsoCode", "USA"), + field( + "player", + selections( + field("name", "Jane Doe") + ) + ) + ) + ) + ) + ).toResponse(); + + runQueryWithExpectedResult(graphQLRequest, expected); + } + + /** + * Test invalid where promotion on a dimension field that is not grouped. + * @throws Exception exception + */ + @Test + public void ungroupedHavingDimensionTest() throws Exception { + String graphQLRequest = document( + selection( + field( + "playerStats", + arguments( + argument("filter", "\"countryIsoCode==\\\"USA\\\",lowScore<\\\"45\\\"\"") + ), + selections( + field("lowScore") + ) + ) + ) + ).toQuery(); + + String errorMessage = "\"Exception while fetching data (/playerStats) : Invalid operation: " + + "'Dimension field countryIsoCode must be grouped before filtering in having clause.'\""; + + runQueryWithExpectedError(graphQLRequest, errorMessage); + } + + /** + * Test invalid having clause on a metric field that is not aggregated. + * @throws Exception exception + */ + @Test + public void nonAggregatedHavingMetricTest() throws Exception { + String graphQLRequest = document( + selection( + field( + "playerStats", + arguments( + argument("filter", "\"highScore<\\\"45\\\"\"") + ), + selections( + field("lowScore") + ) + ) + ) + ).toQuery(); + + String errorMessage = "\"Exception while fetching data (/playerStats) : Invalid operation: " + + "'Metric field highScore must be aggregated before filtering in having clause.'\""; + + runQueryWithExpectedError(graphQLRequest, errorMessage); + } + + /** + * Test invalid where promotion on a different class than the queried class. + * @throws Exception exception + */ + @Test + public void invalidHavingClauseClassTest() throws Exception { + String graphQLRequest = document( + selection( + field( + "playerStats", + arguments( + argument("filter", "\"country.isoCode==\\\"USA\\\",lowScore<\\\"45\\\"\"") + ), + selections( + field("lowScore") + ) + ) + ) + ).toQuery(); + + String errorMessage = "\"Exception while fetching data (/playerStats) : Invalid operation: " + + "'Can't filter on relationship field [PlayerStats].country/[Country].isoCode in HAVING clause " + + "when querying table PlayerStats.'\""; + + runQueryWithExpectedError(graphQLRequest, errorMessage); + } + + @Test + public void dimensionSortingTest() throws Exception { + String graphQLRequest = document( + selection( + field( + "playerStats", + arguments( + argument("sort", "\"overallRating\"") + ), + selections( + field("lowScore"), + field("overallRating") + ) + ) + ) + ).toQuery(); + + String expected = document( + selections( + field( + "playerStats", + selections( + field("lowScore", 35), + field("overallRating", "Good") + ), + selections( + field("lowScore", 241), + field("overallRating", "Great") + ) + ) + ) + ).toResponse(); + + runQueryWithExpectedResult(graphQLRequest, expected); + } + + @Test + public void metricSortingTest() throws Exception { + String graphQLRequest = document( + selection( + field( + "playerStats", + arguments( + argument("sort", "\"-highScore\"") + ), + selections( + field("highScore"), + field( + "country", + selections( + field("name") + ) + ) + ) + ) + ) + ).toQuery(); + + String expected = document( + selections( + field( + "playerStats", + selections( + field("highScore", 2412), + field( + "country", + selections( + field("name", "United States") + ) + ) + ), + selections( + field("highScore", 1000), + field( + "country", + selections( + field("name", "Hong Kong") + ) + ) + ) + ) + ) + ).toResponse(); + + runQueryWithExpectedResult(graphQLRequest, expected); + } + + @Test + public void multipleColumnsSortingTest() throws Exception { + String graphQLRequest = document( + selection( + field( + "playerStats", + arguments( + argument("sort", "\"overallRating,player.name\"") + ), + selections( + field("lowScore"), + field("overallRating"), + field( + "player", + selections( + field("name") + ) + ) + ) + ) + ) + ).toQuery(); + + String expected = document( + selections( + field( + "playerStats", + selections( + field("lowScore", 72), + field("overallRating", "Good"), + field( + "player", + selections( + field("name", "Han") + ) + ) + ), + selections( + field("lowScore", 35), + field("overallRating", "Good"), + field( + "player", + selections( + field("name", "Jon Doe") + ) + ) + ), + selections( + field("lowScore", 241), + field("overallRating", "Great"), + field( + "player", + selections( + field("name", "Jane Doe") + ) + ) + ) + ) + ) + ).toResponse(); + + runQueryWithExpectedResult(graphQLRequest, expected); + } + + @Test + public void idSortingTest() throws Exception { + String graphQLRequest = document( + selection( + field( + "playerStats", + arguments( + argument("sort", "\"id\"") + ), + selections( + field("lowScore"), + field("id") + ) + ) + ) + ).toQuery(); + + String expected = "\"Exception while fetching data (/playerStats) : Invalid operation: 'Sorting on id field is not permitted'\""; + + runQueryWithExpectedError(graphQLRequest, expected); + } + + @Test + public void nestedDimensionNotInQuerySortingTest() throws Exception { + String graphQLRequest = document( + selection( + field( + "playerStats", + arguments( + argument("sort", "\"-country.name,lowScore\"") + ), + selections( + field("lowScore") + ) + ) + ) + ).toQuery(); + + String expected = "\"Exception while fetching data (/playerStats) : Invalid operation: 'Can't sort on country as it is not present in query'\""; + + runQueryWithExpectedError(graphQLRequest, expected); + } + + @Test + public void sortingOnMetricNotInQueryTest() throws Exception { + String graphQLRequest = document( + selection( + field( + "playerStats", + arguments( + argument("sort", "\"highScore\"") + ), + selections( + field("lowScore"), + field( + "country", + selections( + field("name") + ) + ) + ) + ) + ) + ).toQuery(); + + String expected = "\"Exception while fetching data (/playerStats) : Invalid operation: 'Can't sort on highScore as it is not present in query'\""; + + runQueryWithExpectedError(graphQLRequest, expected); + } + + @Test + public void sortingMultipleLevelNesting() throws Exception { + String graphQLRequest = document( + selection( + field( + "playerStats", + arguments( + argument("sort", "\"country.continent.name\"") + ), + selections( + field("lowScore"), + field( + "country", + selections( + field("name"), + field( + "continent", + selections( + field("name") + ) + ) + ) + ) + ) + ) + ) + ).toQuery(); + + String expected = "\"Exception while fetching data (/playerStats) : Currently sorting on double nested fields is not supported\""; + + runQueryWithExpectedError(graphQLRequest, expected); + } + + @Test + @Disabled + //FIXME Needs metric computation support for test case to be valid. + public void aggregationComputedMetricTest() throws Exception { + String graphQLRequest = document( + selection( + field( + "videoGame", + selections( + field("timeSpent"), + field("sessions"), + field("timeSpentPerSession"), + field("timeSpentPerGame") + ) + ) + ) + ).toQuery(); + + String expected = document( + selections( + field( + "videoGame", + selections( + field("timeSpent", 1400), + field("sessions", 70), + field("timeSpentPerSession", 20), + field("timeSpentPerGame", 14) + ) + ) + ) + ).toResponse(); + + runQueryWithExpectedResult(graphQLRequest, expected); + } + + @Test + public void basicViewAggregationTest() throws Exception { + String graphQLRequest = document( + selection( + field( + "playerStatsWithView", + selections( + field("highScore"), + field( + "country", + selections( + field("name") + ) + ), + field("countryViewIsoCode") + ) + ) + ) + ).toQuery(); + + String expected = document( + selections( + field( + "playerStatsWithView", + selections( + field("highScore", 1000), + field( + "country", + selections( + field("name", "Hong Kong") + ) + ), + field("countryViewIsoCode", "HKG") + ), + selections( + field("highScore", 2412), + field( + "country", + selections( + field("name", "United States") + ) + ), + field("countryViewIsoCode", "USA") + ) + ) + ) + ).toResponse(); + + runQueryWithExpectedResult(graphQLRequest, expected); + } + + @Test + public void jsonApiAggregationTest() { + given() + .accept("application/vnd.api+json") + .get("/playerStats") + .then() + .statusCode(HttpStatus.SC_OK) + .body("data.id", hasItems("0", "1", "2")) + .body("data.attributes.highScore", hasItems(1000, 1234, 2412)) + .body("data.relationships.country.data.id", hasItems("840", "344")); + } + + @Test + public void metaDataTest() { + given() + .accept("application/vnd.api+json") + .get("/table/country") + .then() + .statusCode(HttpStatus.SC_OK) + .body("data.attributes.cardinality", equalTo("SMALL")) + .body("data.relationships.columns.data.id", hasItems("country.id", "country.name", "country.isoCode")); + + given() + .accept("application/vnd.api+json") + .get("/analyticView/playerStats") + .then() + .statusCode(HttpStatus.SC_OK) + .body("data.attributes.cardinality", equalTo("LARGE")) + .body( + "data.relationships.dimensions.data.id", + hasItems( + "playerStats.id", + "playerStats.player", + "playerStats.country", + "playerStats.subCountry", + "playerStats.recordedDate", + "playerStats.overallRating", + "playerStats.countryIsoCode", + "playerStats.subCountryIsoCode")) + .body("data.relationships.metrics.data.id", hasItems("playerStats.lowScore", "playerStats.highScore")); + + given() + .accept("application/vnd.api+json") + .get("/dimension/playerStats.player") + .then() + .statusCode(HttpStatus.SC_OK) + .body("data.attributes.name", equalTo("player")) + .body("data.attributes.tableName", equalTo("playerStats")) + .body("data.relationships.dataType.data.id", equalTo("player")); + + given() + .accept("application/vnd.api+json") + .get("/metric/playerStats.lowScore?include=metricFunction") + .then() + .statusCode(HttpStatus.SC_OK) + .body("data.attributes.name", equalTo("lowScore")) + .body("data.attributes.tableName", equalTo("playerStats")) + .body("data.relationships.dataType.data.id", equalTo("p_bigint")) + .body("data.relationships.metricFunction.data.id", equalTo("playerStats.lowScore[min]")) + .body("included.id", hasItem("playerStats.lowScore[min]")) + .body("included.attributes.description", hasItem("sql min function")) + .body("included.attributes.expression", hasItem("MIN(lowScore)")) + .body("included.attributes.longName", hasItem("min")); + + given() + .accept("application/vnd.api+json") + .get("/timeDimension/playerStats.recordedDate?include=supportedGrains") + .then() + .statusCode(HttpStatus.SC_OK) + .body("data.attributes.name", equalTo("recordedDate")) + .body("data.attributes.tableName", equalTo("playerStats")) + .body("data.relationships.dataType.data.id", equalTo("date")) + .body( + "data.relationships.supportedGrains.data.id", + hasItems("playerStats.recordedDate.day", "playerStats.recordedDate.month")) + .body("included.id", hasItems("playerStats.recordedDate.day", "playerStats.recordedDate.month")) + .body("included.attributes.grain", hasItems("DAY", "MONTH")) + .body( + "included.attributes.expression", + hasItems( + "PARSEDATETIME(FORMATDATETIME(%s, 'yyyy-MM-dd'), 'yyyy-MM-dd')", + "PARSEDATETIME(FORMATDATETIME(%s, 'yyyy-MM-01'), 'yyyy-MM-dd')")); + } + + private void create(String query, Map variables) throws IOException { + runQuery(toJsonQuery(query, variables)); + } + + private void runQueryWithExpectedResult( + String graphQLQuery, + Map variables, + String expected + ) throws IOException { + compareJsonObject(runQuery(graphQLQuery, variables), expected); + } + + private void runQueryWithExpectedResult(String graphQLQuery, String expected) throws IOException { + runQueryWithExpectedResult(graphQLQuery, null, expected); + } + + private void runQueryWithExpectedError( + String graphQLQuery, + Map variables, + String errorMessage + ) throws IOException { + compareErrorMessage(runQuery(graphQLQuery, variables), errorMessage); + } + + private void runQueryWithExpectedError(String graphQLQuery, String errorMessage) throws IOException { + runQueryWithExpectedError(graphQLQuery, null, errorMessage); + } + + private void compareJsonObject(ValidatableResponse response, String expected) throws IOException { + JsonNode responseNode = JSON_MAPPER.readTree(response.extract().body().asString()); + JsonNode expectedNode = JSON_MAPPER.readTree(expected); + assertEquals(expectedNode, responseNode); + } + + private void compareErrorMessage(ValidatableResponse response, String expected) throws IOException { + JsonNode responseNode = JSON_MAPPER.readTree(response.extract().body().asString()); + assertEquals(expected, responseNode.get("errors").get(0).get("message").toString()); + } + + private ValidatableResponse runQuery(String query, Map variables) throws IOException { + return runQuery(toJsonQuery(query, variables)); + } + + private ValidatableResponse runQuery(String query) { + ValidatableResponse res = given() + .contentType(MediaType.APPLICATION_JSON) + .accept(MediaType.APPLICATION_JSON) + .body(query) + .post("/graphQL") + .then(); + + return res; + } + + private String toJsonArray(JsonNode... nodes) throws IOException { + ArrayNode arrayNode = JsonNodeFactory.instance.arrayNode(); + for (JsonNode node : nodes) { + arrayNode.add(node); + } + return JSON_MAPPER.writeValueAsString(arrayNode); + } + + private String toJsonQuery(String query, Map variables) throws IOException { + return JSON_MAPPER.writeValueAsString(toJsonNode(query, variables)); + } + + private JsonNode toJsonNode(String query) { + return toJsonNode(query, null); + } + + private JsonNode toJsonNode(String query, Map variables) { + ObjectNode graphqlNode = JsonNodeFactory.instance.objectNode(); + graphqlNode.put("query", query); + if (variables != null) { + graphqlNode.set("variables", JSON_MAPPER.valueToTree(variables)); + } + return graphqlNode; + } +} diff --git a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/metadata/MetaDataStoreTest.java b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/metadata/MetaDataStoreTest.java new file mode 100644 index 0000000000..673112b9ea --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/metadata/MetaDataStoreTest.java @@ -0,0 +1,47 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.metadata; + +import static org.junit.jupiter.api.Assertions.assertNotNull; + +import com.yahoo.elide.core.EntityDictionary; +import com.yahoo.elide.datastores.aggregation.example.Country; +import com.yahoo.elide.datastores.aggregation.example.CountryView; +import com.yahoo.elide.datastores.aggregation.example.CountryViewNested; +import com.yahoo.elide.datastores.aggregation.example.Player; +import com.yahoo.elide.datastores.aggregation.example.PlayerStats; +import com.yahoo.elide.datastores.aggregation.example.PlayerStatsView; +import com.yahoo.elide.datastores.aggregation.example.PlayerStatsWithView; +import com.yahoo.elide.datastores.aggregation.example.SubCountry; + +import org.junit.jupiter.api.BeforeAll; +import org.junit.jupiter.api.Test; + +import java.util.HashMap; + +public class MetaDataStoreTest { + private static MetaDataStore dataStore = new MetaDataStore(); + + @BeforeAll + public static void setup() { + EntityDictionary dictionary = new EntityDictionary(new HashMap<>()); + dictionary.bindEntity(PlayerStatsWithView.class); + dictionary.bindEntity(PlayerStatsView.class); + dictionary.bindEntity(PlayerStats.class); + dictionary.bindEntity(Country.class); + dictionary.bindEntity(SubCountry.class); + dictionary.bindEntity(Player.class); + dictionary.bindEntity(CountryView.class); + dictionary.bindEntity(CountryViewNested.class); + + dataStore.populateEntityDictionary(dictionary); + } + + @Test + public void testSetup() { + assertNotNull(dataStore); + } +} diff --git a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/QueryEngineTest.java b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/QueryEngineTest.java new file mode 100644 index 0000000000..6ba6d1f548 --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/QueryEngineTest.java @@ -0,0 +1,681 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ + +package com.yahoo.elide.datastores.aggregation.queryengines.sql; + +import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.junit.jupiter.api.Assertions.assertNotNull; + +import com.yahoo.elide.core.Path; +import com.yahoo.elide.core.filter.FilterPredicate; +import com.yahoo.elide.core.filter.Operator; +import com.yahoo.elide.core.pagination.Pagination; +import com.yahoo.elide.core.sort.Sorting; +import com.yahoo.elide.datastores.aggregation.example.PlayerStats; +import com.yahoo.elide.datastores.aggregation.example.PlayerStatsView; +import com.yahoo.elide.datastores.aggregation.framework.SQLUnitTest; +import com.yahoo.elide.datastores.aggregation.metadata.models.AnalyticView; +import com.yahoo.elide.datastores.aggregation.query.Query; +import com.yahoo.elide.datastores.aggregation.queryengines.sql.metadata.SQLAnalyticView; +import com.yahoo.elide.datastores.aggregation.time.TimeGrain; + +import com.google.common.collect.Lists; +import org.junit.jupiter.api.BeforeAll; +import org.junit.jupiter.api.Test; + +import java.sql.Timestamp; +import java.util.List; +import java.util.Map; +import java.util.TreeMap; +import java.util.stream.Collectors; +import java.util.stream.StreamSupport; + +public class QueryEngineTest extends SQLUnitTest { + private static AnalyticView playerStatsViewTable; + + @BeforeAll + public static void init() { + SQLUnitTest.init(); + + playerStatsViewTable = new SQLAnalyticView(PlayerStatsView.class, dictionary); + } + + /** + * Test loading all three records from the table. + */ + @Test + public void testFullTableLoad() { + Query query = Query.builder() + .analyticView(playerStatsTable) + .metric(invoke(playerStatsTable.getMetric("lowScore"))) + .metric(invoke(playerStatsTable.getMetric("highScore"))) + .timeDimension(toProjection(playerStatsTable.getTimeDimension("recordedDate"), TimeGrain.DAY)) + .build(); + + List results = StreamSupport.stream(engine.executeQuery(query).spliterator(), false) + .collect(Collectors.toList()); + + PlayerStats stats0 = new PlayerStats(); + stats0.setId("0"); + stats0.setLowScore(241); + stats0.setHighScore(2412); + stats0.setRecordedDate(Timestamp.valueOf("2019-07-11 00:00:00")); + + PlayerStats stats1 = new PlayerStats(); + stats1.setId("1"); + stats1.setLowScore(35); + stats1.setHighScore(1234); + stats1.setRecordedDate(Timestamp.valueOf("2019-07-12 00:00:00")); + + PlayerStats stats2 = new PlayerStats(); + stats2.setId("2"); + stats2.setLowScore(72); + stats2.setHighScore(1000); + stats2.setRecordedDate(Timestamp.valueOf("2019-07-13 00:00:00")); + + assertEquals(3, results.size()); + assertEquals(stats0, results.get(0)); + assertEquals(stats1, results.get(1)); + assertEquals(stats2, results.get(2)); + } + + /** + * Test group by a degenerate dimension with a filter applied. + * + * @throws Exception exception + */ + @Test + public void testDegenerateDimensionFilter() throws Exception { + Query query = Query.builder() + .analyticView(playerStatsTable) + .metric(invoke(playerStatsTable.getMetric("lowScore"))) + .groupByDimension(toProjection(playerStatsTable.getDimension("overallRating"))) + .timeDimension(toProjection(playerStatsTable.getTimeDimension("recordedDate"), TimeGrain.DAY)) + .whereFilter(filterParser.parseFilterExpression("overallRating==Great", + PlayerStats.class, false)) + .build(); + + List results = StreamSupport.stream(engine.executeQuery(query).spliterator(), false) + .collect(Collectors.toList()); + + PlayerStats stats1 = new PlayerStats(); + stats1.setId("0"); + stats1.setLowScore(241); + stats1.setOverallRating("Great"); + stats1.setRecordedDate(Timestamp.valueOf("2019-07-11 00:00:00")); + + assertEquals(1, results.size()); + assertEquals(stats1, results.get(0)); + } + + /** + * Test filtering on a dimension attribute. + * + * @throws Exception exception + */ + @Test + public void testFilterJoin() throws Exception { + Query query = Query.builder() + .analyticView(playerStatsTable) + .metric(invoke(playerStatsTable.getMetric("lowScore"))) + .groupByDimension(toProjection(playerStatsTable.getDimension("country"))) + .whereFilter(filterParser.parseFilterExpression("country.name=='United States'", + PlayerStats.class, false)) + .build(); + + List results = StreamSupport.stream(engine.executeQuery(query).spliterator(), false) + .collect(Collectors.toList()); + + PlayerStats usa0 = new PlayerStats(); + usa0.setId("0"); + usa0.setLowScore(35); + usa0.setCountry(USA); + + assertEquals(1, results.size()); + assertEquals(usa0, results.get(0)); + + // test relationship hydration + PlayerStats actualStats1 = (PlayerStats) results.get(0); + assertNotNull(actualStats1.getCountry()); + } + + /** + * Test filtering on an attribute that's not present in the query. + * + * @throws Exception exception + */ + @Test + public void testSubqueryFilterJoin() throws Exception { + Query query = Query.builder() + .analyticView(playerStatsViewTable) + .metric(invoke(playerStatsTable.getMetric("highScore"))) + .whereFilter(filterParser.parseFilterExpression("player.name=='Jane Doe'", + PlayerStatsView.class, false)) + .build(); + + List results = StreamSupport.stream(engine.executeQuery(query).spliterator(), false) + .collect(Collectors.toList()); + + PlayerStatsView stats2 = new PlayerStatsView(); + stats2.setId("0"); + stats2.setHighScore(2412); + + assertEquals(1, results.size()); + assertEquals(stats2, results.get(0)); + } + + /** + * Test a view which filters on "stats.overallRating = 'Great'". + */ + @Test + public void testSubqueryLoad() { + Query query = Query.builder() + .analyticView(playerStatsViewTable) + .metric(invoke(playerStatsTable.getMetric("highScore"))) + .build(); + + List results = StreamSupport.stream(engine.executeQuery(query).spliterator(), false) + .collect(Collectors.toList()); + + PlayerStatsView stats2 = new PlayerStatsView(); + stats2.setId("0"); + stats2.setHighScore(2412); + + assertEquals(1, results.size()); + assertEquals(stats2, results.get(0)); + } + + /** + * Test sorting by dimension attribute which is not present in the query. + */ + @Test + public void testSortJoin() { + Map sortMap = new TreeMap<>(); + sortMap.put("player.name", Sorting.SortOrder.asc); + + Query query = Query.builder() + .analyticView(playerStatsTable) + .metric(invoke(playerStatsTable.getMetric("lowScore"))) + .groupByDimension(toProjection(playerStatsTable.getDimension("overallRating"))) + .timeDimension(toProjection(playerStatsTable.getTimeDimension("recordedDate"), TimeGrain.DAY)) + .sorting(new Sorting(sortMap)) + .build(); + + List results = StreamSupport.stream(engine.executeQuery(query).spliterator(), false) + .collect(Collectors.toList()); + + PlayerStats stats0 = new PlayerStats(); + stats0.setId("0"); + stats0.setLowScore(72); + stats0.setOverallRating("Good"); + stats0.setRecordedDate(Timestamp.valueOf("2019-07-13 00:00:00")); + + PlayerStats stats1 = new PlayerStats(); + stats1.setId("1"); + stats1.setLowScore(241); + stats1.setOverallRating("Great"); + stats1.setRecordedDate(Timestamp.valueOf("2019-07-11 00:00:00")); + + PlayerStats stats2 = new PlayerStats(); + stats2.setId("2"); + stats2.setLowScore(35); + stats2.setOverallRating("Good"); + stats2.setRecordedDate(Timestamp.valueOf("2019-07-12 00:00:00")); + + assertEquals(3, results.size()); + assertEquals(stats0, results.get(0)); + assertEquals(stats1, results.get(1)); + assertEquals(stats2, results.get(2)); + } + + /** + * Test pagination. + */ + @Test + public void testPagination() { + Pagination pagination = Pagination.fromOffsetAndLimit(1, 0, true); + + Query query = Query.builder() + .analyticView(playerStatsTable) + .metric(invoke(playerStatsTable.getMetric("lowScore"))) + .groupByDimension(toProjection(playerStatsTable.getDimension("overallRating"))) + .timeDimension(toProjection(playerStatsTable.getTimeDimension("recordedDate"), TimeGrain.DAY)) + .pagination(pagination) + .build(); + + List results = StreamSupport.stream(engine.executeQuery(query).spliterator(), false) + .collect(Collectors.toList()); + + //Jon Doe,1234,72,Good,840,2019-07-12 00:00:00 + PlayerStats stats1 = new PlayerStats(); + stats1.setId("0"); + stats1.setLowScore(35); + stats1.setOverallRating("Good"); + stats1.setRecordedDate(Timestamp.valueOf("2019-07-12 00:00:00")); + + assertEquals(results.size(), 1, "Number of records returned does not match"); + assertEquals(results.get(0), stats1, "Returned record does not match"); + assertEquals(pagination.getPageTotals(), 3, "Page totals does not match"); + } + + /** + * Test having clause integrates with group by clause. + * + * @throws Exception exception + */ + @Test + public void testHavingClause() throws Exception { + Query query = Query.builder() + .analyticView(playerStatsTable) + .metric(invoke(playerStatsTable.getMetric("highScore"))) + .groupByDimension(toProjection(playerStatsTable.getDimension("overallRating"))) + .havingFilter(filterParser.parseFilterExpression("highScore < 2400", + PlayerStats.class, false)) + .build(); + + List results = StreamSupport.stream(engine.executeQuery(query).spliterator(), false) + .collect(Collectors.toList()); + + // Only "Good" rating would have total high score less than 2400 + PlayerStats stats1 = new PlayerStats(); + stats1.setId("0"); + stats1.setOverallRating("Good"); + stats1.setHighScore(1234); + + assertEquals(1, results.size()); + assertEquals(stats1, results.get(0)); + } + + /** + * Test having clause integrates with group by clause and join. + * + * @throws Exception exception + */ + @Test + public void testHavingClauseJoin() throws Exception { + Query query = Query.builder() + .analyticView(playerStatsTable) + .metric(invoke(playerStatsTable.getMetric("highScore"))) + .groupByDimension(toProjection(playerStatsTable.getDimension("overallRating"))) + .groupByDimension(toProjection(playerStatsTable.getDimension("countryIsoCode"))) + .havingFilter(filterParser.parseFilterExpression("countryIsoCode==USA", + PlayerStats.class, false)) + .build(); + + List results = StreamSupport.stream(engine.executeQuery(query).spliterator(), false) + .collect(Collectors.toList()); + + PlayerStats stats0 = new PlayerStats(); + stats0.setId("0"); + stats0.setOverallRating("Great"); + stats0.setCountryIsoCode("USA"); + stats0.setHighScore(2412); + + PlayerStats stats1 = new PlayerStats(); + stats1.setId("1"); + stats1.setOverallRating("Good"); + stats1.setCountryIsoCode("USA"); + stats1.setHighScore(1234); + + assertEquals(2, results.size()); + assertEquals(stats0, results.get(0)); + assertEquals(stats1, results.get(1)); + } + + /** + * Test group by, having, dimension, metric at the same time. + * + * @throws Exception exception + */ + @Test + public void testEdgeCasesQuery() throws Exception { + Map sortMap = new TreeMap<>(); + sortMap.put("player.name", Sorting.SortOrder.asc); + + Query query = Query.builder() + .analyticView(playerStatsViewTable) + .metric(invoke(playerStatsTable.getMetric("highScore"))) + .groupByDimension(toProjection(playerStatsViewTable.getDimension("countryName"))) + .whereFilter(filterParser.parseFilterExpression("player.name=='Jane Doe'", + PlayerStatsView.class, false)) + .havingFilter(filterParser.parseFilterExpression("highScore > 300", + PlayerStatsView.class, false)) + .sorting(new Sorting(sortMap)) + .build(); + + List results = StreamSupport.stream(engine.executeQuery(query).spliterator(), false) + .collect(Collectors.toList()); + + PlayerStatsView stats2 = new PlayerStatsView(); + stats2.setId("0"); + stats2.setHighScore(2412); + stats2.setCountryName("United States"); + + assertEquals(1, results.size()); + assertEquals(stats2, results.get(0)); + } + + /** + * Test sorting by two different columns-one metric and one dimension. + */ + @Test + public void testSortByMultipleColumns() { + Map sortMap = new TreeMap<>(); + sortMap.put("lowScore", Sorting.SortOrder.desc); + sortMap.put("player.name", Sorting.SortOrder.asc); + + Query query = Query.builder() + .analyticView(playerStatsTable) + .metric(invoke(playerStatsTable.getMetric("lowScore"))) + .groupByDimension(toProjection(playerStatsTable.getDimension("overallRating"))) + .timeDimension(toProjection(playerStatsTable.getTimeDimension("recordedDate"), TimeGrain.DAY)) + .sorting(new Sorting(sortMap)) + .build(); + + List results = StreamSupport.stream(engine.executeQuery(query).spliterator(), false) + .collect(Collectors.toList()); + + PlayerStats stats0 = new PlayerStats(); + stats0.setId("0"); + stats0.setLowScore(241); + stats0.setOverallRating("Great"); + stats0.setRecordedDate(Timestamp.valueOf("2019-07-11 00:00:00")); + + PlayerStats stats1 = new PlayerStats(); + stats1.setId("1"); + stats1.setLowScore(72); + stats1.setOverallRating("Good"); + stats1.setRecordedDate(Timestamp.valueOf("2019-07-13 00:00:00")); + + PlayerStats stats2 = new PlayerStats(); + stats2.setId("2"); + stats2.setLowScore(35); + stats2.setOverallRating("Good"); + stats2.setRecordedDate(Timestamp.valueOf("2019-07-12 00:00:00")); + + assertEquals(3, results.size()); + assertEquals(stats0, results.get(0)); + assertEquals(stats1, results.get(1)); + assertEquals(stats2, results.get(2)); + } + + /** + * Test hydrating multiple relationship values. Make sure the objects are constructed correctly. + */ + @Test + public void testRelationshipHydration() { + Map sortMap = new TreeMap<>(); + sortMap.put("country.name", Sorting.SortOrder.desc); + sortMap.put("overallRating", Sorting.SortOrder.desc); + + Query query = Query.builder() + .analyticView(playerStatsTable) + .metric(invoke(playerStatsTable.getMetric("lowScore"))) + .groupByDimension(toProjection(playerStatsTable.getDimension("overallRating"))) + .groupByDimension(toProjection(playerStatsTable.getDimension("country"))) + .sorting(new Sorting(sortMap)) + .build(); + + List results = StreamSupport.stream(engine.executeQuery(query).spliterator(), false) + .collect(Collectors.toList()); + + PlayerStats usa0 = new PlayerStats(); + usa0.setId("0"); + usa0.setLowScore(241); + usa0.setOverallRating("Great"); + usa0.setCountry(USA); + + PlayerStats usa1 = new PlayerStats(); + usa1.setId("1"); + usa1.setLowScore(35); + usa1.setOverallRating("Good"); + usa1.setCountry(USA); + + PlayerStats hk2 = new PlayerStats(); + hk2.setId("2"); + hk2.setLowScore(72); + hk2.setOverallRating("Good"); + hk2.setCountry(HONG_KONG); + + assertEquals(3, results.size()); + assertEquals(usa0, results.get(0)); + assertEquals(usa1, results.get(1)); + assertEquals(hk2, results.get(2)); + + // test join + PlayerStats actualStats1 = (PlayerStats) results.get(0); + assertNotNull(actualStats1.getCountry()); + } + + /** + * Test grouping by a dimension with a JoinTo annotation. + */ + @Test + public void testJoinToGroupBy() { + Query query = Query.builder() + .analyticView(playerStatsTable) + .metric(invoke(playerStatsTable.getMetric("highScore"))) + .groupByDimension(toProjection(playerStatsTable.getDimension("countryIsoCode"))) + .build(); + + List results = StreamSupport.stream(engine.executeQuery(query).spliterator(), false) + .collect(Collectors.toList()); + + PlayerStats stats1 = new PlayerStats(); + stats1.setId("0"); + stats1.setHighScore(2412); + stats1.setCountryIsoCode("USA"); + + PlayerStats stats2 = new PlayerStats(); + stats2.setId("1"); + stats2.setHighScore(1000); + stats2.setCountryIsoCode("HKG"); + + assertEquals(2, results.size()); + assertEquals(stats1, results.get(0)); + assertEquals(stats2, results.get(1)); + } + + /** + * Test grouping by a dimension with a JoinTo annotation. + * + * @throws Exception exception + */ + @Test + public void testJoinToFilter() throws Exception { + Query query = Query.builder() + .analyticView(playerStatsTable) + .metric(invoke(playerStatsTable.getMetric("highScore"))) + .groupByDimension(toProjection(playerStatsTable.getDimension("overallRating"))) + .whereFilter(filterParser.parseFilterExpression("countryIsoCode==USA", + PlayerStats.class, false)) + .build(); + + List results = StreamSupport.stream(engine.executeQuery(query).spliterator(), false) + .collect(Collectors.toList()); + + PlayerStats stats1 = new PlayerStats(); + stats1.setId("0"); + stats1.setOverallRating("Good"); + stats1.setHighScore(1234); + + PlayerStats stats2 = new PlayerStats(); + stats2.setId("1"); + stats2.setOverallRating("Great"); + stats2.setHighScore(2412); + + assertEquals(2, results.size()); + assertEquals(stats1, results.get(0)); + assertEquals(stats2, results.get(1)); + } + + /** + * Test grouping by a dimension with a JoinTo annotation. + */ + @Test + public void testJoinToSort() { + Map sortMap = new TreeMap<>(); + sortMap.put("countryIsoCode", Sorting.SortOrder.asc); + + Query query = Query.builder() + .analyticView(playerStatsTable) + .metric(invoke(playerStatsTable.getMetric("highScore"))) + .groupByDimension(toProjection(playerStatsTable.getDimension("overallRating"))) + .groupByDimension(toProjection(playerStatsTable.getDimension("country"))) + .sorting(new Sorting(sortMap)) + .build(); + + List results = StreamSupport.stream(engine.executeQuery(query).spliterator(), false) + .collect(Collectors.toList()); + + PlayerStats stats1 = new PlayerStats(); + stats1.setId("0"); + stats1.setOverallRating("Good"); + stats1.setCountry(HONG_KONG); + stats1.setHighScore(1000); + + PlayerStats stats2 = new PlayerStats(); + stats2.setId("1"); + stats2.setOverallRating("Good"); + stats2.setCountry(USA); + stats2.setHighScore(1234); + + PlayerStats stats3 = new PlayerStats(); + stats3.setId("2"); + stats3.setOverallRating("Great"); + stats3.setCountry(USA); + stats3.setHighScore(2412); + + assertEquals(3, results.size()); + assertEquals(stats1, results.get(0)); + assertEquals(stats2, results.get(1)); + assertEquals(stats3, results.get(2)); + } + + /** + * Test month grain query. + */ + @Test + public void testTotalScoreByMonth() { + Query query = Query.builder() + .analyticView(playerStatsTable) + .metric(invoke(playerStatsTable.getMetric("highScore"))) + .timeDimension(toProjection(playerStatsTable.getTimeDimension("recordedDate"), TimeGrain.MONTH)) + .build(); + + List results = StreamSupport.stream(engine.executeQuery(query).spliterator(), false) + .collect(Collectors.toList()); + + PlayerStats stats0 = new PlayerStats(); + stats0.setId("0"); + stats0.setHighScore(2412); + stats0.setRecordedDate(Timestamp.valueOf("2019-07-01 00:00:00")); + + assertEquals(1, results.size()); + assertEquals(stats0, results.get(0)); + } + + /** + * Test filter by time dimension. + */ + @Test + public void testFilterByTemporalDimension() { + FilterPredicate predicate = new FilterPredicate( + new Path(PlayerStats.class, dictionary, "recordedDate"), + Operator.IN, + Lists.newArrayList(Timestamp.valueOf("2019-07-11 00:00:00"))); + + Query query = Query.builder() + .analyticView(playerStatsTable) + .metric(invoke(playerStatsTable.getMetric("highScore"))) + .timeDimension(toProjection(playerStatsTable.getTimeDimension("recordedDate"), TimeGrain.DAY)) + .whereFilter(predicate) + .build(); + + List results = StreamSupport.stream(engine.executeQuery(query).spliterator(), false) + .collect(Collectors.toList()); + + PlayerStats stats0 = new PlayerStats(); + stats0.setId("0"); + stats0.setHighScore(2412); + stats0.setRecordedDate(Timestamp.valueOf("2019-07-11 00:00:00")); + + assertEquals(1, results.size()); + assertEquals(stats0, results.get(0)); + } + + @Test + public void testSortAggregatedMetric() { + Map sortMap = new TreeMap<>(); + sortMap.put("lowScore", Sorting.SortOrder.desc); + + Query query = Query.builder() + .analyticView(playerStatsTable) + .groupByDimension(toProjection(playerStatsTable.getDimension("overallRating"))) + .metric(invoke(playerStatsTable.getMetric("lowScore"))) + .sorting(new Sorting(sortMap)) + .build(); + + List results = StreamSupport.stream(engine.executeQuery(query).spliterator(), false) + .collect(Collectors.toList()); + + PlayerStats stats0 = new PlayerStats(); + stats0.setId("0"); + stats0.setLowScore(241); + stats0.setOverallRating("Great"); + + PlayerStats stats1 = new PlayerStats(); + stats1.setId("1"); + stats1.setLowScore(35); + stats1.setOverallRating("Good"); + + assertEquals(2, results.size()); + assertEquals(stats0, results.get(0)); + assertEquals(stats1, results.get(1)); + } + + @Test + public void testAmbiguousFields() { + Map sortMap = new TreeMap<>(); + sortMap.put("lowScore", Sorting.SortOrder.asc); + + Query query = Query.builder() + .analyticView(playerStatsTable) + .groupByDimension(toProjection(playerStatsTable.getDimension("playerName"))) + .groupByDimension(toProjection(playerStatsTable.getDimension("player2Name"))) + .metric(invoke(playerStatsTable.getMetric("lowScore"))) + .sorting(new Sorting(sortMap)) + .build(); + + List results = StreamSupport.stream(engine.executeQuery(query).spliterator(), false) + .collect(Collectors.toList()); + + PlayerStats stats0 = new PlayerStats(); + stats0.setId("0"); + stats0.setLowScore(35); + stats0.setPlayerName("Jon Doe"); + stats0.setPlayer2Name("Jane Doe"); + + PlayerStats stats1 = new PlayerStats(); + stats1.setId("1"); + stats1.setLowScore(72); + stats1.setPlayerName("Han"); + stats1.setPlayer2Name("Jon Doe"); + + PlayerStats stats2 = new PlayerStats(); + stats2.setId("2"); + stats2.setLowScore(241); + stats2.setPlayerName("Jane Doe"); + stats2.setPlayer2Name("Han"); + + assertEquals(3, results.size()); + assertEquals(stats0, results.get(0)); + assertEquals(stats1, results.get(1)); + assertEquals(stats2, results.get(2)); + } + + //TODO - Add Invalid Request Tests +} diff --git a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/SubselectTest.java b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/SubselectTest.java new file mode 100644 index 0000000000..b0f536c9db --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/SubselectTest.java @@ -0,0 +1,267 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ + +package com.yahoo.elide.datastores.aggregation.queryengines.sql; + +import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.junit.jupiter.api.Assertions.assertNotNull; + +import com.yahoo.elide.core.sort.Sorting; +import com.yahoo.elide.datastores.aggregation.example.PlayerStats; +import com.yahoo.elide.datastores.aggregation.example.SubCountry; +import com.yahoo.elide.datastores.aggregation.framework.SQLUnitTest; +import com.yahoo.elide.datastores.aggregation.query.Query; +import com.yahoo.elide.datastores.aggregation.time.TimeGrain; + +import org.junit.jupiter.api.BeforeAll; +import org.junit.jupiter.api.Test; + +import java.sql.Timestamp; +import java.util.List; +import java.util.Map; +import java.util.TreeMap; +import java.util.stream.Collectors; +import java.util.stream.StreamSupport; + +public class SubselectTest extends SQLUnitTest { + private static final SubCountry SUB_HONG_KONG = new SubCountry(); + private static final SubCountry SUB_USA = new SubCountry(); + + @BeforeAll + public static void init() { + SQLUnitTest.init(); + + SUB_HONG_KONG.setIsoCode("HKG"); + SUB_HONG_KONG.setName("Hong Kong"); + SUB_HONG_KONG.setId("344"); + + SUB_USA.setIsoCode("USA"); + SUB_USA.setName("United States"); + SUB_USA.setId("840"); + } + + /** + * Test filtering on a dimension attribute. + * + * @throws Exception exception + */ + @Test + public void testFilterJoin() throws Exception { + Query query = Query.builder() + .analyticView(playerStatsTable) + .metric(invoke(playerStatsTable.getMetric("lowScore"))) + .metric(invoke(playerStatsTable.getMetric("highScore"))) + .groupByDimension(toProjection(playerStatsTable.getDimension("overallRating"))) + .groupByDimension(toProjection(playerStatsTable.getDimension("subCountry"))) + .groupByDimension(toProjection(playerStatsTable.getDimension("country"))) + .timeDimension(toProjection(playerStatsTable.getTimeDimension("recordedDate"), TimeGrain.DAY)) + .whereFilter(filterParser.parseFilterExpression("subCountry.name=='United States'", + PlayerStats.class, false)) + .build(); + + + List results = StreamSupport.stream(engine.executeQuery(query).spliterator(), false) + .collect(Collectors.toList()); + + PlayerStats usa0 = new PlayerStats(); + usa0.setId("0"); + usa0.setLowScore(35); + usa0.setHighScore(1234); + usa0.setOverallRating("Good"); + usa0.setCountry(USA); + usa0.setSubCountry(SUB_USA); + usa0.setRecordedDate(Timestamp.valueOf("2019-07-12 00:00:00")); + + PlayerStats usa1 = new PlayerStats(); + usa1.setId("1"); + usa1.setLowScore(241); + usa1.setHighScore(2412); + usa1.setOverallRating("Great"); + usa1.setCountry(USA); + usa1.setSubCountry(SUB_USA); + usa1.setRecordedDate(Timestamp.valueOf("2019-07-11 00:00:00")); + + assertEquals(2, results.size()); + assertEquals(usa0, results.get(0)); + assertEquals(usa1, results.get(1)); + + // test join + PlayerStats actualStats0 = (PlayerStats) results.get(0); + assertNotNull(actualStats0.getSubCountry()); + assertNotNull(actualStats0.getCountry()); + } + + /** + * Test hydrating multiple relationship values. Make sure the objects are constructed correctly. + */ + @Test + public void testRelationshipHydration() { + Map sortMap = new TreeMap<>(); + sortMap.put("subCountry.name", Sorting.SortOrder.desc); + + Query query = Query.builder() + .analyticView(playerStatsTable) + .metric(invoke(playerStatsTable.getMetric("lowScore"))) + .metric(invoke(playerStatsTable.getMetric("highScore"))) + .groupByDimension(toProjection(playerStatsTable.getDimension("overallRating"))) + .groupByDimension(toProjection(playerStatsTable.getDimension("country"))) + .groupByDimension(toProjection(playerStatsTable.getDimension("subCountry"))) + .timeDimension(toProjection(playerStatsTable.getTimeDimension("recordedDate"), TimeGrain.DAY)) + .sorting(new Sorting(sortMap)) + .build(); + + List results = StreamSupport.stream(engine.executeQuery(query).spliterator(), false) + .collect(Collectors.toList()); + + PlayerStats usa0 = new PlayerStats(); + usa0.setId("0"); + usa0.setLowScore(35); + usa0.setHighScore(1234); + usa0.setOverallRating("Good"); + usa0.setCountry(USA); + usa0.setSubCountry(SUB_USA); + usa0.setRecordedDate(Timestamp.valueOf("2019-07-12 00:00:00")); + + PlayerStats usa1 = new PlayerStats(); + usa1.setId("1"); + usa1.setLowScore(241); + usa1.setHighScore(2412); + usa1.setOverallRating("Great"); + usa1.setCountry(USA); + usa1.setSubCountry(SUB_USA); + usa1.setRecordedDate(Timestamp.valueOf("2019-07-11 00:00:00")); + + PlayerStats hk2 = new PlayerStats(); + hk2.setId("2"); + hk2.setLowScore(72); + hk2.setHighScore(1000); + hk2.setOverallRating("Good"); + hk2.setCountry(HONG_KONG); + hk2.setSubCountry(SUB_HONG_KONG); + hk2.setRecordedDate(Timestamp.valueOf("2019-07-13 00:00:00")); + + assertEquals(3, results.size()); + assertEquals(usa0, results.get(0)); + assertEquals(usa1, results.get(1)); + assertEquals(hk2, results.get(2)); + + // test join + PlayerStats actualStats0 = (PlayerStats) results.get(0); + assertNotNull(actualStats0.getSubCountry()); + assertNotNull(actualStats0.getCountry()); + } + + /** + * Test grouping by a dimension with a JoinTo annotation. + * + * @throws Exception exception + */ + @Test + public void testJoinToGroupBy() throws Exception { + Query query = Query.builder() + .analyticView(playerStatsTable) + .metric(invoke(playerStatsTable.getMetric("highScore"))) + .groupByDimension(toProjection(playerStatsTable.getDimension("subCountryIsoCode"))) + .build(); + + List results = StreamSupport.stream(engine.executeQuery(query).spliterator(), false) + .collect(Collectors.toList()); + + PlayerStats stats1 = new PlayerStats(); + stats1.setId("0"); + stats1.setHighScore(2412); + stats1.setSubCountryIsoCode("USA"); + + PlayerStats stats2 = new PlayerStats(); + stats2.setId("1"); + stats2.setHighScore(1000); + stats2.setSubCountryIsoCode("HKG"); + + assertEquals(2, results.size()); + assertEquals(stats1, results.get(0)); + assertEquals(stats2, results.get(1)); + } + + /** + * Test grouping by a dimension with a JoinTo annotation. + * + * @throws Exception exception + */ + @Test + public void testJoinToFilter() throws Exception { + Query query = Query.builder() + .analyticView(playerStatsTable) + .metric(invoke(playerStatsTable.getMetric("highScore"))) + .groupByDimension(toProjection(playerStatsTable.getDimension("overallRating"))) + .whereFilter(filterParser.parseFilterExpression("subCountryIsoCode==USA", + PlayerStats.class, false)) + .build(); + + List results = StreamSupport.stream(engine.executeQuery(query).spliterator(), false) + .collect(Collectors.toList()); + + PlayerStats stats1 = new PlayerStats(); + stats1.setId("0"); + stats1.setOverallRating("Good"); + stats1.setHighScore(1234); + + PlayerStats stats2 = new PlayerStats(); + stats2.setId("1"); + stats2.setOverallRating("Great"); + stats2.setHighScore(2412); + + assertEquals(2, results.size()); + assertEquals(stats1, results.get(0)); + assertEquals(stats2, results.get(1)); + } + + /** + * Test grouping by a dimension with a JoinTo annotation. + * + * @throws Exception exception + */ + @Test + public void testJoinToSort() throws Exception { + Map sortMap = new TreeMap<>(); + sortMap.put("subCountryIsoCode", Sorting.SortOrder.asc); + + Query query = Query.builder() + .analyticView(playerStatsTable) + .metric(invoke(playerStatsTable.getMetric("highScore"))) + .groupByDimension(toProjection(playerStatsTable.getDimension("overallRating"))) + .groupByDimension(toProjection(playerStatsTable.getDimension("subCountry"))) + .sorting(new Sorting(sortMap)) + .build(); + + List results = StreamSupport.stream(engine.executeQuery(query).spliterator(), false) + .collect(Collectors.toList()); + + PlayerStats stats1 = new PlayerStats(); + stats1.setId("0"); + stats1.setOverallRating("Good"); + stats1.setSubCountry(SUB_HONG_KONG); + stats1.setHighScore(1000); + + PlayerStats stats2 = new PlayerStats(); + stats2.setId("1"); + stats2.setOverallRating("Good"); + stats2.setSubCountry(SUB_USA); + stats2.setHighScore(1234); + + PlayerStats stats3 = new PlayerStats(); + stats3.setId("2"); + stats3.setOverallRating("Great"); + stats3.setSubCountry(SUB_USA); + stats3.setHighScore(2412); + + assertEquals(3, results.size()); + assertEquals(stats1, results.get(0)); + assertEquals(stats2, results.get(1)); + assertEquals(stats3, results.get(2)); + } + + //TODO - Add Invalid Request Tests +} diff --git a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/ViewTest.java b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/ViewTest.java new file mode 100644 index 0000000000..779029104a --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/ViewTest.java @@ -0,0 +1,261 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ + +package com.yahoo.elide.datastores.aggregation.queryengines.sql; + +import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.junit.jupiter.api.Assertions.assertNull; +import static org.junit.jupiter.api.Assertions.assertThrows; + +import com.yahoo.elide.core.exceptions.InvalidPredicateException; +import com.yahoo.elide.core.sort.Sorting; +import com.yahoo.elide.datastores.aggregation.example.PlayerStatsWithView; +import com.yahoo.elide.datastores.aggregation.framework.SQLUnitTest; +import com.yahoo.elide.datastores.aggregation.metadata.models.AnalyticView; +import com.yahoo.elide.datastores.aggregation.query.Query; +import com.yahoo.elide.datastores.aggregation.queryengines.sql.metadata.SQLAnalyticView; + +import org.junit.jupiter.api.BeforeAll; +import org.junit.jupiter.api.Test; + +import java.util.List; +import java.util.Map; +import java.util.TreeMap; +import java.util.stream.Collectors; +import java.util.stream.StreamSupport; + +public class ViewTest extends SQLUnitTest { + protected static AnalyticView playerStatsWithViewSchema; + + @BeforeAll + public static void init() { + SQLUnitTest.init(); + playerStatsWithViewSchema = new SQLAnalyticView(PlayerStatsWithView.class, dictionary); + } + + @Test + public void testViewRelationFailure() { + Map sortMap = new TreeMap<>(); + sortMap.put("countryViewIsoCode", Sorting.SortOrder.desc); + + Query query = Query.builder() + .analyticView(playerStatsWithViewSchema) + .metric(invoke(playerStatsWithViewSchema.getMetric("lowScore"))) + .groupByDimension(toProjection(playerStatsWithViewSchema.getDimension("countryView"))) + .sorting(new Sorting(sortMap)) + .build(); + + assertThrows(InvalidPredicateException.class, () -> engine.executeQuery(query)); + } + + @Test + public void testViewAttribute() { + Map sortMap = new TreeMap<>(); + sortMap.put("countryViewIsoCode", Sorting.SortOrder.desc); + + Query query = Query.builder() + .analyticView(playerStatsWithViewSchema) + .metric(invoke(playerStatsWithViewSchema.getMetric("lowScore"))) + .groupByDimension(toProjection(playerStatsWithViewSchema.getDimension("countryViewIsoCode"))) + .sorting(new Sorting(sortMap)) + .build(); + + List results = StreamSupport.stream(engine.executeQuery(query).spliterator(), false) + .collect(Collectors.toList()); + + PlayerStatsWithView usa0 = new PlayerStatsWithView(); + usa0.setId("0"); + usa0.setLowScore(35); + usa0.setCountryViewIsoCode("USA"); + + PlayerStatsWithView hk1 = new PlayerStatsWithView(); + hk1.setId("1"); + hk1.setLowScore(72); + hk1.setCountryViewIsoCode("HKG"); + + assertEquals(2, results.size()); + assertEquals(usa0, results.get(0)); + assertEquals(hk1, results.get(1)); + + // the join would not happen for a view join + PlayerStatsWithView actualStats1 = (PlayerStatsWithView) results.get(0); + assertNull(actualStats1.getCountry()); + } + + @Test + public void testNestedViewAttribute() throws Exception { + Map sortMap = new TreeMap<>(); + sortMap.put("countryViewIsoCode", Sorting.SortOrder.desc); + + Query query = Query.builder() + .analyticView(playerStatsWithViewSchema) + .metric(invoke(playerStatsWithViewSchema.getMetric("lowScore"))) + .groupByDimension(toProjection(playerStatsWithViewSchema.getDimension("countryViewViewIsoCode"))) + .sorting(new Sorting(sortMap)) + .build(); + + List results = StreamSupport.stream(engine.executeQuery(query).spliterator(), false) + .collect(Collectors.toList()); + + PlayerStatsWithView usa0 = new PlayerStatsWithView(); + usa0.setId("0"); + usa0.setLowScore(35); + usa0.setCountryViewViewIsoCode("USA"); + + PlayerStatsWithView hk1 = new PlayerStatsWithView(); + hk1.setId("1"); + hk1.setLowScore(72); + hk1.setCountryViewViewIsoCode("HKG"); + + assertEquals(2, results.size()); + assertEquals(usa0, results.get(0)); + assertEquals(hk1, results.get(1)); + + // the join would not happen for a view join + PlayerStatsWithView actualStats1 = (PlayerStatsWithView) results.get(0); + assertNull(actualStats1.getCountry()); + } + + @Test + public void testNestedRelationshipAttribute() throws Exception { + Map sortMap = new TreeMap<>(); + sortMap.put("countryViewIsoCode", Sorting.SortOrder.desc); + + Query query = Query.builder() + .analyticView(playerStatsWithViewSchema) + .metric(invoke(playerStatsWithViewSchema.getMetric("lowScore"))) + .groupByDimension( + toProjection(playerStatsWithViewSchema.getDimension("countryViewRelationshipIsoCode"))) + .sorting(new Sorting(sortMap)) + .build(); + + List results = StreamSupport.stream(engine.executeQuery(query).spliterator(), false) + .collect(Collectors.toList()); + + PlayerStatsWithView usa0 = new PlayerStatsWithView(); + usa0.setId("0"); + usa0.setLowScore(35); + usa0.setCountryViewRelationshipIsoCode("USA"); + + PlayerStatsWithView hk1 = new PlayerStatsWithView(); + hk1.setId("1"); + hk1.setLowScore(72); + hk1.setCountryViewRelationshipIsoCode("HKG"); + + assertEquals(2, results.size()); + assertEquals(usa0, results.get(0)); + assertEquals(hk1, results.get(1)); + + // the join would not happen for a view join + PlayerStatsWithView actualStats1 = (PlayerStatsWithView) results.get(0); + assertNull(actualStats1.getCountry()); + } + + @Test + public void testSortingViewAttribute() throws Exception { + Map sortMap = new TreeMap<>(); + sortMap.put("countryView.isoCode", Sorting.SortOrder.desc); + + Query query = Query.builder() + .analyticView(playerStatsWithViewSchema) + .metric(invoke(playerStatsWithViewSchema.getMetric("lowScore"))) + .groupByDimension( + toProjection(playerStatsWithViewSchema.getDimension("countryViewRelationshipIsoCode"))) + .sorting(new Sorting(sortMap)) + .build(); + + List results = StreamSupport.stream(engine.executeQuery(query).spliterator(), false) + .collect(Collectors.toList()); + + PlayerStatsWithView usa0 = new PlayerStatsWithView(); + usa0.setId("0"); + usa0.setLowScore(35); + usa0.setCountryViewRelationshipIsoCode("USA"); + + PlayerStatsWithView hk1 = new PlayerStatsWithView(); + hk1.setId("1"); + hk1.setLowScore(72); + hk1.setCountryViewRelationshipIsoCode("HKG"); + + assertEquals(2, results.size()); + assertEquals(usa0, results.get(0)); + assertEquals(hk1, results.get(1)); + + // the join would not happen for a view join + PlayerStatsWithView actualStats1 = (PlayerStatsWithView) results.get(0); + assertNull(actualStats1.getCountry()); + } + + @Test + public void testSortingNestedViewAttribute() throws Exception { + Map sortMap = new TreeMap<>(); + sortMap.put("countryView.nestedView.isoCode", Sorting.SortOrder.desc); + + Query query = Query.builder() + .analyticView(playerStatsWithViewSchema) + .metric(invoke(playerStatsWithViewSchema.getMetric("lowScore"))) + .groupByDimension( + toProjection(playerStatsWithViewSchema.getDimension("countryViewRelationshipIsoCode"))) + .sorting(new Sorting(sortMap)) + .build(); + + List results = StreamSupport.stream(engine.executeQuery(query).spliterator(), false) + .collect(Collectors.toList()); + + PlayerStatsWithView usa0 = new PlayerStatsWithView(); + usa0.setId("0"); + usa0.setLowScore(35); + usa0.setCountryViewRelationshipIsoCode("USA"); + + PlayerStatsWithView hk1 = new PlayerStatsWithView(); + hk1.setId("1"); + hk1.setLowScore(72); + hk1.setCountryViewRelationshipIsoCode("HKG"); + + assertEquals(2, results.size()); + assertEquals(usa0, results.get(0)); + assertEquals(hk1, results.get(1)); + + // the join would not happen for a view join + PlayerStatsWithView actualStats1 = (PlayerStatsWithView) results.get(0); + assertNull(actualStats1.getCountry()); + } + + @Test + public void testSortingNestedRelationshipAttribute() throws Exception { + Map sortMap = new TreeMap<>(); + sortMap.put("countryView.nestedRelationship.isoCode", Sorting.SortOrder.desc); + + Query query = Query.builder() + .analyticView(playerStatsWithViewSchema) + .metric(invoke(playerStatsWithViewSchema.getMetric("lowScore"))) + .groupByDimension( + toProjection(playerStatsWithViewSchema.getDimension("countryViewRelationshipIsoCode"))) + .sorting(new Sorting(sortMap)) + .build(); + + List results = StreamSupport.stream(engine.executeQuery(query).spliterator(), false) + .collect(Collectors.toList()); + + PlayerStatsWithView usa0 = new PlayerStatsWithView(); + usa0.setId("0"); + usa0.setLowScore(35); + usa0.setCountryViewRelationshipIsoCode("USA"); + + PlayerStatsWithView hk1 = new PlayerStatsWithView(); + hk1.setId("1"); + hk1.setLowScore(72); + hk1.setCountryViewRelationshipIsoCode("HKG"); + + assertEquals(2, results.size()); + assertEquals(usa0, results.get(0)); + assertEquals(hk1, results.get(1)); + + // the join would not happen for a view join + PlayerStatsWithView actualStats1 = (PlayerStatsWithView) results.get(0); + assertNull(actualStats1.getCountry()); + } +} diff --git a/elide-datastore/elide-datastore-aggregation/src/test/resources/META-INF/persistence.xml b/elide-datastore/elide-datastore-aggregation/src/test/resources/META-INF/persistence.xml new file mode 100644 index 0000000000..2844a0b784 --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/test/resources/META-INF/persistence.xml @@ -0,0 +1,28 @@ + + + + + + com.yahoo.elide.datastores.aggregation.example.Country + com.yahoo.elide.datastores.aggregation.example.SubCountry + com.yahoo.elide.datastores.aggregation.example.Player + com.yahoo.elide.datastores.aggregation.example.Continent + + + + + + + + + + + + diff --git a/elide-datastore/elide-datastore-aggregation/src/test/resources/continent.csv b/elide-datastore/elide-datastore-aggregation/src/test/resources/continent.csv new file mode 100644 index 0000000000..b9f40f2ced --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/test/resources/continent.csv @@ -0,0 +1,3 @@ +id,name +1,Asia +2,North America diff --git a/elide-datastore/elide-datastore-aggregation/src/test/resources/country.csv b/elide-datastore/elide-datastore-aggregation/src/test/resources/country.csv new file mode 100644 index 0000000000..ae0eeebc02 --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/test/resources/country.csv @@ -0,0 +1,3 @@ +id,isoCode,name,continent_id +344,HKG,Hong Kong,1 +840,USA,United States,2 diff --git a/elide-datastore/elide-datastore-aggregation/src/test/resources/create_tables.sql b/elide-datastore/elide-datastore-aggregation/src/test/resources/create_tables.sql new file mode 100644 index 0000000000..1667f6f061 --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/test/resources/create_tables.sql @@ -0,0 +1,37 @@ +CREATE TABLE IF NOT EXISTS playerStats + ( + highScore BIGINT, + lowScore BIGINT, + overallRating VARCHAR(255), + country_id VARCHAR(255), + sub_country_id VARCHAR(255), + player_id BIGINT, + player2_id BIGINT, + recordedDate DATETIME + ) AS SELECT * FROM CSVREAD('classpath:player_stats.csv'); + +CREATE TABLE IF NOT EXISTS countries + ( + id VARCHAR(255), + isoCode VARCHAR(255), + name VARCHAR(255), + continent_id VARCHAR(255) + ) AS SELECT * FROM CSVREAD('classpath:country.csv'); + +CREATE TABLE IF NOT EXISTS players + ( + id BIGINT, + name VARCHAR(255) + ) AS SELECT * FROM CSVREAD('classpath:player.csv'); + +CREATE TABLE IF NOT EXISTS videoGames + ( + game_rounds BIGINT, + timeSpent BIGINT + ) AS SELECT * FROM CSVREAD('classpath:video_games.csv'); + +CREATE TABLE IF NOT EXISTS continents + ( + id BIGINT, + name VARCHAR(255) + ) AS SELECT * FROM CSVREAD('classpath:continent.csv'); diff --git a/elide-datastore/elide-datastore-aggregation/src/test/resources/player.csv b/elide-datastore/elide-datastore-aggregation/src/test/resources/player.csv new file mode 100644 index 0000000000..68b871ea30 --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/test/resources/player.csv @@ -0,0 +1,4 @@ +id,name +1,Jon Doe +2,Jane Doe +3,Han diff --git a/elide-datastore/elide-datastore-aggregation/src/test/resources/player_stats.csv b/elide-datastore/elide-datastore-aggregation/src/test/resources/player_stats.csv new file mode 100644 index 0000000000..0778a7aa53 --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/test/resources/player_stats.csv @@ -0,0 +1,4 @@ +highScore,lowScore,overallRating,country_id,sub_country_id,player_id,player2_id,recordedDate +1234,35,Good,840,840,1,2,2019-07-12 00:00:00 +2412,241,Great,840,840,2,3,2019-07-11 00:00:00 +1000,72,Good,344,344,3,1,2019-07-13 00:00:00 diff --git a/elide-datastore/elide-datastore-aggregation/src/test/resources/video_games.csv b/elide-datastore/elide-datastore-aggregation/src/test/resources/video_games.csv new file mode 100644 index 0000000000..407c57c551 --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/test/resources/video_games.csv @@ -0,0 +1,5 @@ +rounds,timeSpent +10,50 +20,150 +30,1000 +10,200 diff --git a/elide-datastore/elide-datastore-hibernate/pom.xml b/elide-datastore/elide-datastore-hibernate/pom.xml index ef226d743a..c50501afc0 100644 --- a/elide-datastore/elide-datastore-hibernate/pom.xml +++ b/elide-datastore/elide-datastore-hibernate/pom.xml @@ -14,7 +14,7 @@ com.yahoo.elide elide-datastore-parent-pom - 4.6.2-SNAPSHOT + 5.0.0-pr6-SNAPSHOT @@ -58,7 +58,7 @@ com.yahoo.elide elide-integration-tests - 4.6.2-SNAPSHOT + 5.0.0-pr6-SNAPSHOT test-jar test diff --git a/elide-datastore/elide-datastore-hibernate/src/main/java/com/yahoo/elide/core/filter/FilterTranslator.java b/elide-datastore/elide-datastore-hibernate/src/main/java/com/yahoo/elide/core/filter/FilterTranslator.java index f436b1c0d0..5609d4369e 100644 --- a/elide-datastore/elide-datastore-hibernate/src/main/java/com/yahoo/elide/core/filter/FilterTranslator.java +++ b/elide-datastore/elide-datastore-hibernate/src/main/java/com/yahoo/elide/core/filter/FilterTranslator.java @@ -41,6 +41,7 @@ import java.util.HashMap; import java.util.List; import java.util.Map; +import java.util.function.Function; import java.util.stream.Collectors; /** @@ -52,6 +53,11 @@ public class FilterTranslator implements FilterOperation { private static Map operatorGenerators; private static Map, String>, JPQLPredicateGenerator> predicateOverrides; + public static final Function GENERATE_HQL_COLUMN_NO_ALIAS = FilterPredicate::getFieldPath; + + public static final Function GENERATE_HQL_COLUMN_WITH_ALIAS = + (predicate) -> FilterPredicate.getPathAlias(predicate.getPath()) + "." + predicate.getField(); + static { predicateOverrides = new HashMap<>(); @@ -204,8 +210,8 @@ public static void registerJPQLGenerator(Operator op, * @return Returns null if no generator is registered. */ public static JPQLPredicateGenerator lookupJPQLGenerator(Operator op, - Class entityClass, - String fieldName) { + Class entityClass, + String fieldName) { return predicateOverrides.get(Triple.of(op, entityClass, fieldName)); } @@ -225,22 +231,17 @@ public static JPQLPredicateGenerator lookupJPQLGenerator(Operator op) { */ @Override public String apply(FilterPredicate filterPredicate) { - return apply(filterPredicate, false); + return apply(filterPredicate, GENERATE_HQL_COLUMN_NO_ALIAS); } /** * Transforms a filter predicate into a JPQL query fragment. * @param filterPredicate The predicate to transform. - * @param prefixWithAlias Whether or not to append the entity type to the predicate. - * This is useful for table aliases referenced in JPQL for some kinds of joins. + * @param columnGenerator Function which supplies a HQL fragment which represents the column in the predicate. * @return The hql query fragment. */ - protected String apply(FilterPredicate filterPredicate, boolean prefixWithAlias) { - String fieldPath = filterPredicate.getFieldPath(); - - if (prefixWithAlias) { - fieldPath = filterPredicate.getAlias() + "." + filterPredicate.getField(); - } + protected String apply(FilterPredicate filterPredicate, Function columnGenerator) { + String fieldPath = columnGenerator.apply(filterPredicate); Path.PathElement last = filterPredicate.getPath().lastElement().get(); @@ -275,24 +276,46 @@ private static String leastClause(List params) { .collect(Collectors.joining(COMMA))); } + /** + * Translates the filterExpression to a JPQL filter fragment. + * @param filterExpression The filterExpression to translate + * @param prefixWithAlias If true, use the default alias provider to append the predicates with an alias. + * Otherwise, don't append aliases. + * @return A JPQL filter fragment. + */ public String apply(FilterExpression filterExpression, boolean prefixWithAlias) { - JPQLQueryVisitor visitor = new JPQLQueryVisitor(prefixWithAlias); - return "WHERE " + filterExpression.accept(visitor); + Function columnGenerator = GENERATE_HQL_COLUMN_NO_ALIAS; + if (prefixWithAlias) { + columnGenerator = GENERATE_HQL_COLUMN_WITH_ALIAS; + } + + return apply(filterExpression, columnGenerator); + } + + /** + * Translates the filterExpression to a JPQL filter fragment. + * @param filterExpression The filterExpression to translate + * @param columnGenerator Generates a HQL fragment that represents a column in the predicate + * @return A JPQL filter fragment. + */ + public String apply(FilterExpression filterExpression, Function columnGenerator) { + JPQLQueryVisitor visitor = new JPQLQueryVisitor(columnGenerator); + return filterExpression.accept(visitor); } /** * Filter expression visitor which builds an JPQL query. */ public class JPQLQueryVisitor implements FilterExpressionVisitor { - private boolean prefixWithAlias; + private Function columnGenerator; - public JPQLQueryVisitor(boolean prefixWithAlias) { - this.prefixWithAlias = prefixWithAlias; + public JPQLQueryVisitor(Function columnGenerator) { + this.columnGenerator = columnGenerator; } @Override public String visitPredicate(FilterPredicate filterPredicate) { - return apply(filterPredicate, prefixWithAlias); + return apply(filterPredicate, columnGenerator); } @Override diff --git a/elide-datastore/elide-datastore-hibernate/src/main/java/com/yahoo/elide/core/hibernate/hql/AbstractHQLQueryBuilder.java b/elide-datastore/elide-datastore-hibernate/src/main/java/com/yahoo/elide/core/hibernate/hql/AbstractHQLQueryBuilder.java index b8679e71e7..d3ff968df9 100644 --- a/elide-datastore/elide-datastore-hibernate/src/main/java/com/yahoo/elide/core/hibernate/hql/AbstractHQLQueryBuilder.java +++ b/elide-datastore/elide-datastore-hibernate/src/main/java/com/yahoo/elide/core/hibernate/hql/AbstractHQLQueryBuilder.java @@ -5,6 +5,9 @@ */ package com.yahoo.elide.core.hibernate.hql; +import static com.yahoo.elide.core.filter.FilterPredicate.appendAlias; +import static com.yahoo.elide.core.filter.FilterPredicate.getTypeAlias; + import com.yahoo.elide.core.EntityDictionary; import com.yahoo.elide.core.Path; import com.yahoo.elide.core.RelationshipType; @@ -49,6 +52,7 @@ public abstract class AbstractHQLQueryBuilder { protected static final String SELECT = "SELECT "; protected static final String AS = " AS "; protected static final String DISTINCT = "DISTINCT "; + protected static final String WHERE = " WHERE "; protected static final boolean USE_ALIAS = true; protected static final boolean NO_ALIAS = false; @@ -154,23 +158,22 @@ private String extractJoinClause(FilterPredicate predicate, Set alreadyJ for (Path.PathElement pathElement : predicate.getPath().getPathElements()) { String fieldName = pathElement.getFieldName(); Class typeClass = dictionary.lookupEntityClass(pathElement.getType()); - String typeAlias = FilterPredicate.getTypeAlias(typeClass); + String typeAlias = getTypeAlias(typeClass); - //Nothing left to join. + // Nothing left to join. if (! dictionary.isRelation(pathElement.getType(), fieldName)) { return joinClause.toString(); } - String alias = typeAlias + UNDERSCORE + fieldName; + String alias = previousAlias == null + ? appendAlias(typeAlias, fieldName) + : appendAlias(previousAlias, fieldName); - String joinFragment; + String joinFragment = previousAlias == null + ? LEFT + JOIN + typeAlias + PERIOD + fieldName + SPACE + alias + SPACE + : LEFT + JOIN + previousAlias + PERIOD + fieldName + SPACE + alias + SPACE; //This is the first path element - if (previousAlias == null) { - joinFragment = LEFT + JOIN + typeAlias + PERIOD + fieldName + SPACE + alias + SPACE; - } else { - joinFragment = LEFT + JOIN + previousAlias + PERIOD + fieldName + SPACE + alias + SPACE; - } if (!alreadyJoined.contains(joinFragment)) { joinClause.append(joinFragment); diff --git a/elide-datastore/elide-datastore-hibernate/src/main/java/com/yahoo/elide/core/hibernate/hql/RootCollectionFetchQueryBuilder.java b/elide-datastore/elide-datastore-hibernate/src/main/java/com/yahoo/elide/core/hibernate/hql/RootCollectionFetchQueryBuilder.java index c3a33a29e7..3364356b82 100644 --- a/elide-datastore/elide-datastore-hibernate/src/main/java/com/yahoo/elide/core/hibernate/hql/RootCollectionFetchQueryBuilder.java +++ b/elide-datastore/elide-datastore-hibernate/src/main/java/com/yahoo/elide/core/hibernate/hql/RootCollectionFetchQueryBuilder.java @@ -45,7 +45,7 @@ public Query build() { Collection predicates = filterExpression.get().accept(extractor); //Build the WHERE clause - String filterClause = new FilterTranslator().apply(filterExpression.get(), USE_ALIAS); + String filterClause = WHERE + new FilterTranslator().apply(filterExpression.get(), USE_ALIAS); //Build the JOIN clause String joinClause = getJoinClauseFromFilters(filterExpression.get()) diff --git a/elide-datastore/elide-datastore-hibernate/src/main/java/com/yahoo/elide/core/hibernate/hql/RootCollectionPageTotalsQueryBuilder.java b/elide-datastore/elide-datastore-hibernate/src/main/java/com/yahoo/elide/core/hibernate/hql/RootCollectionPageTotalsQueryBuilder.java index de95cead5d..80be6043e3 100644 --- a/elide-datastore/elide-datastore-hibernate/src/main/java/com/yahoo/elide/core/hibernate/hql/RootCollectionPageTotalsQueryBuilder.java +++ b/elide-datastore/elide-datastore-hibernate/src/main/java/com/yahoo/elide/core/hibernate/hql/RootCollectionPageTotalsQueryBuilder.java @@ -67,10 +67,10 @@ public Query build() { predicates = filterExpression.get().accept(extractor); //Build the WHERE clause - filterClause = new FilterTranslator().apply(filterExpression.get(), USE_ALIAS); + filterClause = WHERE + new FilterTranslator().apply(filterExpression.get(), USE_ALIAS); //Build the JOIN clause - joinClause = getJoinClauseFromFilters(filterExpression.get()); + joinClause = getJoinClauseFromFilters(filterExpression.get()); } else { predicates = new HashSet<>(); diff --git a/elide-datastore/elide-datastore-hibernate/src/main/java/com/yahoo/elide/core/hibernate/hql/SubCollectionFetchQueryBuilder.java b/elide-datastore/elide-datastore-hibernate/src/main/java/com/yahoo/elide/core/hibernate/hql/SubCollectionFetchQueryBuilder.java index c0df3d8c13..0dc27fa248 100644 --- a/elide-datastore/elide-datastore-hibernate/src/main/java/com/yahoo/elide/core/hibernate/hql/SubCollectionFetchQueryBuilder.java +++ b/elide-datastore/elide-datastore-hibernate/src/main/java/com/yahoo/elide/core/hibernate/hql/SubCollectionFetchQueryBuilder.java @@ -82,7 +82,7 @@ public Query build() { + JOIN + parentAlias + PERIOD + relationshipName + SPACE + childAlias + joinClause - + SPACE + + WHERE + filterClause + " AND " + parentAlias + "=:" + parentAlias + SPACE diff --git a/elide-datastore/elide-datastore-hibernate/src/main/java/com/yahoo/elide/core/hibernate/hql/SubCollectionPageTotalsQueryBuilder.java b/elide-datastore/elide-datastore-hibernate/src/main/java/com/yahoo/elide/core/hibernate/hql/SubCollectionPageTotalsQueryBuilder.java index 9957ce526e..fb999db7d2 100644 --- a/elide-datastore/elide-datastore-hibernate/src/main/java/com/yahoo/elide/core/hibernate/hql/SubCollectionPageTotalsQueryBuilder.java +++ b/elide-datastore/elide-datastore-hibernate/src/main/java/com/yahoo/elide/core/hibernate/hql/SubCollectionPageTotalsQueryBuilder.java @@ -130,7 +130,7 @@ public Query build() { + parentAlias + SPACE + joinClause - + SPACE + + WHERE + filterClause); //Fill in the query parameters diff --git a/elide-datastore/elide-datastore-hibernate/src/test/java/com/yahoo/elide/core/filter/FilterTranslatorTest.java b/elide-datastore/elide-datastore-hibernate/src/test/java/com/yahoo/elide/core/filter/FilterTranslatorTest.java index 9cc2ebe808..570318a49d 100644 --- a/elide-datastore/elide-datastore-hibernate/src/test/java/com/yahoo/elide/core/filter/FilterTranslatorTest.java +++ b/elide-datastore/elide-datastore-hibernate/src/test/java/com/yahoo/elide/core/filter/FilterTranslatorTest.java @@ -65,6 +65,7 @@ public void testHQLQueryVisitor() throws Exception { FilterTranslator filterOp = new FilterTranslator(); String query = filterOp.apply(not, false); + query = query.trim().replaceAll(" +", " "); String p1Params = p1.getParameters().stream() .map(FilterPredicate.FilterParameter::getPlaceholder).collect(Collectors.joining(", ")); @@ -72,7 +73,7 @@ public void testHQLQueryVisitor() throws Exception { .map(FilterPredicate.FilterParameter::getPlaceholder).collect(Collectors.joining(", ")); String p3Params = p3.getParameters().stream() .map(FilterPredicate.FilterParameter::getPlaceholder).collect(Collectors.joining(", ")); - String expected = "WHERE NOT (((name IN (" + p2Params + ") OR genre IN (" + p3Params + ")) " + String expected = "NOT (((name IN (" + p2Params + ") OR genre IN (" + p3Params + ")) " + "AND (authors IS NOT EMPTY AND authors.name IN (" + p1Params + "))))"; assertEquals(expected, query); } diff --git a/elide-datastore/elide-datastore-hibernate/src/test/java/com/yahoo/elide/datastores/hibernate/hql/AbstractHQLQueryBuilderTest.java b/elide-datastore/elide-datastore-hibernate/src/test/java/com/yahoo/elide/datastores/hibernate/hql/AbstractHQLQueryBuilderTest.java index 64edcd3cf7..f0d5da6832 100644 --- a/elide-datastore/elide-datastore-hibernate/src/test/java/com/yahoo/elide/datastores/hibernate/hql/AbstractHQLQueryBuilderTest.java +++ b/elide-datastore/elide-datastore-hibernate/src/test/java/com/yahoo/elide/datastores/hibernate/hql/AbstractHQLQueryBuilderTest.java @@ -99,8 +99,8 @@ public void testFilterJoinClause() { String actual = getJoinClauseFromFilters(andExpression); String expected = " LEFT JOIN example_Author.books example_Author_books " - + "LEFT JOIN example_Author_books.chapters example_Book_chapters " - + "LEFT JOIN example_Author_books.publisher example_Book_publisher "; + + "LEFT JOIN example_Author_books.chapters example_Author_books_chapters " + + "LEFT JOIN example_Author_books.publisher example_Author_books_publisher "; assertEquals(expected, actual); } diff --git a/elide-datastore/elide-datastore-hibernate/src/test/java/com/yahoo/elide/datastores/hibernate/hql/RootCollectionFetchQueryBuilderTest.java b/elide-datastore/elide-datastore-hibernate/src/test/java/com/yahoo/elide/datastores/hibernate/hql/RootCollectionFetchQueryBuilderTest.java index fb9bd1c386..42682fbe4f 100644 --- a/elide-datastore/elide-datastore-hibernate/src/test/java/com/yahoo/elide/datastores/hibernate/hql/RootCollectionFetchQueryBuilderTest.java +++ b/elide-datastore/elide-datastore-hibernate/src/test/java/com/yahoo/elide/datastores/hibernate/hql/RootCollectionFetchQueryBuilderTest.java @@ -62,6 +62,7 @@ public void testRootFetch() { String expected = "SELECT example_Book FROM example.Book AS example_Book LEFT JOIN FETCH example_Book.publisher "; String actual = query.getQueryText(); + actual = actual.trim().replaceAll(" +", " "); assertEquals(expected, actual); } @@ -81,6 +82,7 @@ public void testRootFetchWithSorting() { String expected = "SELECT example_Book FROM example.Book AS example_Book " + "LEFT JOIN FETCH example_Book.publisher order by example_Book.title asc"; String actual = query.getQueryText(); + actual = actual.trim().replaceAll(" +", " "); assertEquals(expected, actual); } @@ -145,6 +147,7 @@ public void testDistinctRootFetchWithToManyJoinFilterAndPagination() throws Pars + "OR example_Book_publisher.name IN (:books_publisher_name_XXX)) "; String actual = query.getQueryText(); + actual = actual.trim().replaceAll(" +", " "); actual = actual.replaceFirst(":books_chapters_title_\\w\\w\\w\\w+", ":books_chapters_title_XXX"); actual = actual.replaceFirst(":books_chapters_title_\\w\\w\\w\\w+", ":books_chapters_title_XXX"); actual = actual.replaceFirst(":books_publisher_name_\\w\\w\\w\\w+", ":books_publisher_name_XXX"); @@ -189,10 +192,16 @@ public void testRootFetchWithSortingAndFilters() { .build(); String expected = +<<<<<<< HEAD "SELECT example_Book FROM example.Book AS example_Book LEFT JOIN FETCH example_Book.publisher" + " WHERE example_Book.id IN (:id_XXX) order by example_Book.title asc"; +======= + "SELECT example_Book FROM example.Book AS example_Book " + + "WHERE example_Book.id IN (:id_XXX) order by example_Book.title asc"; +>>>>>>> 91591898... Create AggregationDataStore module (#845) String actual = query.getQueryText(); + actual = actual.trim().replaceAll(" +", " "); actual = actual.replaceFirst(":id_\\w+", ":id_XXX"); assertEquals(expected, actual); diff --git a/elide-datastore/elide-datastore-hibernate/src/test/java/com/yahoo/elide/datastores/hibernate/hql/RootCollectionPageTotalsQueryBuilderTest.java b/elide-datastore/elide-datastore-hibernate/src/test/java/com/yahoo/elide/datastores/hibernate/hql/RootCollectionPageTotalsQueryBuilderTest.java index 8e174be8b7..3b15499dad 100644 --- a/elide-datastore/elide-datastore-hibernate/src/test/java/com/yahoo/elide/datastores/hibernate/hql/RootCollectionPageTotalsQueryBuilderTest.java +++ b/elide-datastore/elide-datastore-hibernate/src/test/java/com/yahoo/elide/datastores/hibernate/hql/RootCollectionPageTotalsQueryBuilderTest.java @@ -57,10 +57,11 @@ public void testRootFetch() { TestQueryWrapper query = (TestQueryWrapper) builder.build(); String expected = - "SELECT COUNT(DISTINCT example_Book) " - + "FROM example.Book AS example_Book "; + "SELECT COUNT(DISTINCT example_Book) " + + "FROM example.Book AS example_Book"; String actual = query.getQueryText(); + actual = actual.trim().replaceAll(" +", " "); assertEquals(expected, actual); } @@ -116,14 +117,16 @@ public void testRootFetchWithJoinFilter() { .build(); String expected = - "SELECT COUNT(DISTINCT example_Author) FROM example.Author AS example_Author " - + "LEFT JOIN example_Author.books example_Author_books " - + "LEFT JOIN example_Author_books.chapters example_Book_chapters " - + "LEFT JOIN example_Author_books.publisher example_Book_publisher " - + "WHERE (example_Book_chapters.title IN (:books_chapters_title_XXX, :books_chapters_title_XXX) " - + "OR example_Book_publisher.name IN (:books_publisher_name_XXX))"; + "SELECT COUNT(DISTINCT example_Author) FROM example.Author AS example_Author " + + "LEFT JOIN example_Author.books example_Author_books " + + "LEFT JOIN example_Author_books.chapters example_Author_books_chapters " + + "LEFT JOIN example_Author_books.publisher example_Author_books_publisher " + + "WHERE (example_Author_books_chapters.title IN " + + "(:books_chapters_title_XXX, :books_chapters_title_XXX) " + + "OR example_Author_books_publisher.name IN (:books_publisher_name_XXX))"; String actual = query.getQueryText(); + actual = actual.trim().replaceAll(" +", " "); actual = actual.replaceFirst(":books_chapters_title_\\w\\w\\w\\w+", ":books_chapters_title_XXX"); actual = actual.replaceFirst(":books_chapters_title_\\w\\w\\w\\w+", ":books_chapters_title_XXX"); actual = actual.replaceFirst(":books_publisher_name_\\w\\w\\w\\w+", ":books_publisher_name_XXX"); diff --git a/elide-datastore/elide-datastore-hibernate/src/test/java/com/yahoo/elide/datastores/hibernate/hql/SubCollectionFetchQueryBuilderTest.java b/elide-datastore/elide-datastore-hibernate/src/test/java/com/yahoo/elide/datastores/hibernate/hql/SubCollectionFetchQueryBuilderTest.java index 82255037be..26a73687fd 100644 --- a/elide-datastore/elide-datastore-hibernate/src/test/java/com/yahoo/elide/datastores/hibernate/hql/SubCollectionFetchQueryBuilderTest.java +++ b/elide-datastore/elide-datastore-hibernate/src/test/java/com/yahoo/elide/datastores/hibernate/hql/SubCollectionFetchQueryBuilderTest.java @@ -104,6 +104,7 @@ public void testSubCollectionFetchWithSorting() { + "JOIN example_Author__fetch.books example_Book LEFT JOIN FETCH example_Book.publisher " + "WHERE example_Author__fetch=:example_Author__fetch order by example_Book.title asc"; String actual = query.getQueryText(); + actual = actual.trim().replaceAll(" +", " "); assertEquals(expected, actual); } @@ -193,6 +194,7 @@ public void testSubCollectionFetchWithSortingAndFilters() { String actual = query.getQueryText(); actual = actual.replaceFirst(":publisher_name_\\w+", ":publisher_name_XXX"); + actual = actual.trim().replaceAll(" +", " "); assertEquals(expected, actual); } diff --git a/elide-datastore/elide-datastore-hibernate/src/test/java/com/yahoo/elide/datastores/hibernate/hql/SubCollectionPageTotalsQueryBuilderTest.java b/elide-datastore/elide-datastore-hibernate/src/test/java/com/yahoo/elide/datastores/hibernate/hql/SubCollectionPageTotalsQueryBuilderTest.java index 7f15941f6f..c2e78c9198 100644 --- a/elide-datastore/elide-datastore-hibernate/src/test/java/com/yahoo/elide/datastores/hibernate/hql/SubCollectionPageTotalsQueryBuilderTest.java +++ b/elide-datastore/elide-datastore-hibernate/src/test/java/com/yahoo/elide/datastores/hibernate/hql/SubCollectionPageTotalsQueryBuilderTest.java @@ -73,12 +73,13 @@ public void testSubCollectionPageTotals() { .build(); String actual = query.getQueryText(); + actual = actual.trim().replaceAll(" +", " "); actual = actual.replaceFirst(":id_\\w+", ":id_XXX"); String expected = - "SELECT COUNT(DISTINCT example_Author_books) " - + "FROM example.Author AS example_Author " - + "JOIN example_Author.books example_Author_books " + "SELECT COUNT(DISTINCT example_Author_books) " + + "FROM example.Author AS example_Author " + + "JOIN example_Author.books example_Author_books " + "WHERE example_Author.id IN (:id_XXX)"; assertEquals(expected, actual); @@ -139,15 +140,16 @@ public void testSubCollectionPageTotalsWithJoinFilter() { .build(); String expected = - "SELECT COUNT(DISTINCT example_Author_books) " - + "FROM example.Author AS example_Author " - + "LEFT JOIN example_Author.books example_Author_books " - + "LEFT JOIN example_Author_books.publisher example_Book_publisher " - + "WHERE (example_Book_publisher.name IN (:books_publisher_name_XXX) " + "SELECT COUNT(DISTINCT example_Author_books) " + + "FROM example.Author AS example_Author " + + "LEFT JOIN example_Author.books example_Author_books " + + "LEFT JOIN example_Author_books.publisher example_Author_books_publisher " + + "WHERE (example_Author_books_publisher.name IN (:books_publisher_name_XXX) " + "AND example_Author.id IN (:id_XXX))"; String actual = query.getQueryText(); actual = actual.replaceFirst(":books_publisher_name_\\w+", ":books_publisher_name_XXX"); + actual = actual.trim().replaceAll(" +", " "); actual = actual.replaceFirst(":id_\\w+", ":id_XXX"); assertEquals(expected, actual); diff --git a/elide-datastore/elide-datastore-hibernate3/pom.xml b/elide-datastore/elide-datastore-hibernate3/pom.xml index 3c29fac4f9..94e16bc4d2 100644 --- a/elide-datastore/elide-datastore-hibernate3/pom.xml +++ b/elide-datastore/elide-datastore-hibernate3/pom.xml @@ -14,7 +14,7 @@ com.yahoo.elide elide-datastore-parent-pom - 4.6.2-SNAPSHOT + 5.0.0-pr6-SNAPSHOT @@ -52,19 +52,19 @@ com.yahoo.elide elide-datastore-hibernate - 4.6.2-SNAPSHOT + 5.0.0-pr6-SNAPSHOT com.yahoo.elide elide-datastore-hibernate - 4.6.2-SNAPSHOT + 5.0.0-pr6-SNAPSHOT test-jar test com.yahoo.elide elide-integration-tests - 4.6.2-SNAPSHOT + 5.0.0-pr6-SNAPSHOT test-jar test diff --git a/elide-datastore/elide-datastore-hibernate3/src/main/java/com/yahoo/elide/datastores/hibernate3/HibernateTransaction.java b/elide-datastore/elide-datastore-hibernate3/src/main/java/com/yahoo/elide/datastores/hibernate3/HibernateTransaction.java index bab6d350fa..b7acfb4b6a 100644 --- a/elide-datastore/elide-datastore-hibernate3/src/main/java/com/yahoo/elide/datastores/hibernate3/HibernateTransaction.java +++ b/elide-datastore/elide-datastore-hibernate3/src/main/java/com/yahoo/elide/datastores/hibernate3/HibernateTransaction.java @@ -25,6 +25,8 @@ import com.yahoo.elide.core.sort.Sorting; import com.yahoo.elide.datastores.hibernate3.porting.QueryWrapper; import com.yahoo.elide.datastores.hibernate3.porting.SessionWrapper; +import com.yahoo.elide.request.EntityProjection; +import com.yahoo.elide.request.Relationship; import com.yahoo.elide.security.User; import org.hibernate.FlushMode; @@ -114,17 +116,18 @@ public void createObject(Object entity, RequestScope scope) { /** * load a single record with id and filter. * - * @param entityClass class of query object + * @param projection The projection to query * @param id id of the query object - * @param filterExpression FilterExpression contains the predicates * @param scope Request scope associated with specific request */ @Override - public Object loadObject(Class entityClass, + public Object loadObject(EntityProjection projection, Serializable id, - Optional filterExpression, RequestScope scope) { + Class entityClass = projection.getType(); + FilterExpression filterExpression = projection.getFilterExpression(); + try { EntityDictionary dictionary = scope.getDictionary(); Class idType = dictionary.getIdType(entityClass); @@ -139,9 +142,9 @@ public Object loadObject(Class entityClass, idExpression = new FalsePredicate(idPath); } - FilterExpression joinedExpression = filterExpression - .map(fe -> (FilterExpression) new AndFilterExpression(fe, idExpression)) - .orElse(idExpression); + FilterExpression joinedExpression = (filterExpression != null) + ? new AndFilterExpression(filterExpression, idExpression) + : idExpression; QueryWrapper query = (QueryWrapper) new RootCollectionFetchQueryBuilder(entityClass, dictionary, sessionWrapper) @@ -156,23 +159,24 @@ public Object loadObject(Class entityClass, @Override public Iterable loadObjects( - Class entityClass, - Optional filterExpression, - Optional sorting, - Optional pagination, + EntityProjection projection, RequestScope scope) { - pagination.ifPresent(p -> { - if (p.isGenerateTotals()) { - p.setPageTotals(getTotalRecords(entityClass, filterExpression, scope.getDictionary())); - } - }); + Class entityClass = projection.getType(); + Pagination pagination = projection.getPagination(); + FilterExpression filterExpression = projection.getFilterExpression(); + Sorting sorting = projection.getSorting(); + + if (pagination != null && pagination.isGenerateTotals()) { + pagination.setPageTotals(getTotalRecords(entityClass, + Optional.ofNullable(filterExpression), scope.getDictionary())); + } final QueryWrapper query = (QueryWrapper) new RootCollectionFetchQueryBuilder(entityClass, scope.getDictionary(), sessionWrapper) - .withPossibleFilterExpression(filterExpression) - .withPossibleSorting(sorting) - .withPossiblePagination(pagination) + .withPossibleFilterExpression(Optional.ofNullable(filterExpression)) + .withPossibleSorting(Optional.ofNullable(sorting)) + .withPossiblePagination(Optional.ofNullable(pagination)) .build(); if (isScrollEnabled) { @@ -185,14 +189,15 @@ public Iterable loadObjects( public Object getRelation( DataStoreTransaction relationTx, Object entity, - String relationName, - Optional filterExpression, - Optional sorting, - Optional pagination, + Relationship relation, RequestScope scope) { + FilterExpression filterExpression = relation.getProjection().getFilterExpression(); + Sorting sorting = relation.getProjection().getSorting(); + Pagination pagination = relation.getProjection().getPagination(); + EntityDictionary dictionary = scope.getDictionary(); - Object val = com.yahoo.elide.core.PersistentResource.getValue(entity, relationName, scope); + Object val = com.yahoo.elide.core.PersistentResource.getValue(entity, relation.getName(), scope); if (val instanceof Collection) { Collection filteredVal = (Collection) val; if (filteredVal instanceof AbstractPersistentCollection) { @@ -201,31 +206,30 @@ public Object getRelation( * If there is no filtering or sorting required in the data store, and the pagination is default, * return the proxy and let Hibernate manage the SQL generation. */ - if (! filterExpression.isPresent() && ! sorting.isPresent() - && (! pagination.isPresent() || (pagination.isPresent() && pagination.get().isDefaultInstance()))) { + if (filterExpression == null && sorting == null + && (pagination == null || (pagination.isDefaultInstance()))) { return val; } - Class relationClass = dictionary.getParameterizedType(entity, relationName); + Class relationClass = dictionary.getParameterizedType(entity, relation.getName()); RelationshipImpl relationship = new RelationshipImpl( dictionary.lookupEntityClass(entity.getClass()), relationClass, - relationName, + relation.getName(), entity, filteredVal); - pagination.ifPresent(p -> { - if (p.isGenerateTotals()) { - p.setPageTotals(getTotalRecords(relationship, filterExpression, dictionary)); - } - }); + if (pagination != null && pagination.isGenerateTotals()) { + pagination.setPageTotals(getTotalRecords(relationship, + Optional.ofNullable(filterExpression), scope.getDictionary())); + } final QueryWrapper query = (QueryWrapper) new SubCollectionFetchQueryBuilder(relationship, dictionary, sessionWrapper) - .withPossibleFilterExpression(filterExpression) - .withPossibleSorting(sorting) - .withPossiblePagination(pagination) + .withPossibleFilterExpression(Optional.ofNullable(filterExpression)) + .withPossibleSorting(Optional.ofNullable(sorting)) + .withPossiblePagination(Optional.ofNullable(pagination)) .build(); if (query != null) { diff --git a/elide-datastore/elide-datastore-hibernate5/pom.xml b/elide-datastore/elide-datastore-hibernate5/pom.xml index 528872809b..445b673b68 100644 --- a/elide-datastore/elide-datastore-hibernate5/pom.xml +++ b/elide-datastore/elide-datastore-hibernate5/pom.xml @@ -14,7 +14,7 @@ com.yahoo.elide elide-datastore-parent-pom - 4.6.2-SNAPSHOT + 5.0.0-pr6-SNAPSHOT @@ -51,19 +51,19 @@ com.yahoo.elide elide-datastore-hibernate - 4.6.2-SNAPSHOT + 5.0.0-pr6-SNAPSHOT com.yahoo.elide elide-datastore-hibernate - 4.6.2-SNAPSHOT + 5.0.0-pr6-SNAPSHOT test-jar test com.yahoo.elide elide-integration-tests - 4.6.2-SNAPSHOT + 5.0.0-pr6-SNAPSHOT test-jar test diff --git a/elide-datastore/elide-datastore-hibernate5/src/main/java/com/yahoo/elide/datastores/hibernate5/HibernateTransaction.java b/elide-datastore/elide-datastore-hibernate5/src/main/java/com/yahoo/elide/datastores/hibernate5/HibernateTransaction.java index a8d5c1aa28..f0e08aeac5 100644 --- a/elide-datastore/elide-datastore-hibernate5/src/main/java/com/yahoo/elide/datastores/hibernate5/HibernateTransaction.java +++ b/elide-datastore/elide-datastore-hibernate5/src/main/java/com/yahoo/elide/datastores/hibernate5/HibernateTransaction.java @@ -25,6 +25,8 @@ import com.yahoo.elide.core.sort.Sorting; import com.yahoo.elide.datastores.hibernate5.porting.QueryWrapper; import com.yahoo.elide.datastores.hibernate5.porting.SessionWrapper; +import com.yahoo.elide.request.EntityProjection; +import com.yahoo.elide.request.Relationship; import com.yahoo.elide.security.User; import org.hibernate.FlushMode; @@ -119,17 +121,18 @@ public void createObject(Object entity, RequestScope scope) { /** * load a single record with id and filter. * - * @param entityClass class of query object + * @param projection The projection to query * @param id id of the query object - * @param filterExpression FilterExpression contains the predicates * @param scope Request scope associated with specific request */ @Override - public Object loadObject(Class entityClass, + public Object loadObject(EntityProjection projection, Serializable id, - Optional filterExpression, RequestScope scope) { + Class entityClass = projection.getType(); + FilterExpression filterExpression = projection.getFilterExpression(); + try { EntityDictionary dictionary = scope.getDictionary(); Class idType = dictionary.getIdType(entityClass); @@ -144,9 +147,9 @@ public Object loadObject(Class entityClass, idExpression = new FalsePredicate(idPath); } - FilterExpression joinedExpression = filterExpression - .map(fe -> (FilterExpression) new AndFilterExpression(fe, idExpression)) - .orElse(idExpression); + FilterExpression joinedExpression = (filterExpression != null) + ? new AndFilterExpression(filterExpression, idExpression) + : idExpression; QueryWrapper query = (QueryWrapper) new RootCollectionFetchQueryBuilder(entityClass, dictionary, sessionWrapper) @@ -161,23 +164,24 @@ public Object loadObject(Class entityClass, @Override public Iterable loadObjects( - Class entityClass, - Optional filterExpression, - Optional sorting, - Optional pagination, + EntityProjection projection, RequestScope scope) { - pagination.ifPresent(p -> { - if (p.isGenerateTotals()) { - p.setPageTotals(getTotalRecords(entityClass, filterExpression, scope.getDictionary())); - } - }); + Class entityClass = projection.getType(); + Pagination pagination = projection.getPagination(); + FilterExpression filterExpression = projection.getFilterExpression(); + Sorting sorting = projection.getSorting(); + + if (pagination != null && pagination.isGenerateTotals()) { + pagination.setPageTotals(getTotalRecords(entityClass, + Optional.ofNullable(filterExpression), scope.getDictionary())); + } final QueryWrapper query = (QueryWrapper) new RootCollectionFetchQueryBuilder(entityClass, scope.getDictionary(), sessionWrapper) - .withPossibleFilterExpression(filterExpression) - .withPossibleSorting(sorting) - .withPossiblePagination(pagination) + .withPossibleFilterExpression(Optional.ofNullable(filterExpression)) + .withPossibleSorting(Optional.ofNullable(sorting)) + .withPossiblePagination(Optional.ofNullable(pagination)) .build(); @@ -191,14 +195,15 @@ public Iterable loadObjects( public Object getRelation( DataStoreTransaction relationTx, Object entity, - String relationName, - Optional filterExpression, - Optional sorting, - Optional pagination, + Relationship relation, RequestScope scope) { + FilterExpression filterExpression = relation.getProjection().getFilterExpression(); + Sorting sorting = relation.getProjection().getSorting(); + Pagination pagination = relation.getProjection().getPagination(); + EntityDictionary dictionary = scope.getDictionary(); - Object val = com.yahoo.elide.core.PersistentResource.getValue(entity, relationName, scope); + Object val = com.yahoo.elide.core.PersistentResource.getValue(entity, relation.getName(), scope); if (val instanceof Collection) { Collection filteredVal = (Collection) val; if (filteredVal instanceof AbstractPersistentCollection) { @@ -207,31 +212,30 @@ public Object getRelation( * If there is no filtering or sorting required in the data store, and the pagination is default, * return the proxy and let Hibernate manage the SQL generation. */ - if (! filterExpression.isPresent() && ! sorting.isPresent() - && (! pagination.isPresent() || (pagination.isPresent() && pagination.get().isDefaultInstance()))) { + if (filterExpression == null && sorting == null + && (pagination == null || (pagination.isDefaultInstance()))) { return val; } - Class relationClass = dictionary.getParameterizedType(entity, relationName); + Class relationClass = dictionary.getParameterizedType(entity, relation.getName()); RelationshipImpl relationship = new RelationshipImpl( dictionary.lookupEntityClass(entity.getClass()), relationClass, - relationName, + relation.getName(), entity, filteredVal); - pagination.ifPresent(p -> { - if (p.isGenerateTotals()) { - p.setPageTotals(getTotalRecords(relationship, filterExpression, dictionary)); - } - }); + if (pagination != null && pagination.isGenerateTotals()) { + pagination.setPageTotals(getTotalRecords(relationship, + Optional.ofNullable(filterExpression), scope.getDictionary())); + } final QueryWrapper query = (QueryWrapper) new SubCollectionFetchQueryBuilder(relationship, dictionary, sessionWrapper) - .withPossibleFilterExpression(filterExpression) - .withPossibleSorting(sorting) - .withPossiblePagination(pagination) + .withPossibleFilterExpression(Optional.ofNullable(filterExpression)) + .withPossibleSorting(Optional.ofNullable(sorting)) + .withPossiblePagination(Optional.ofNullable(pagination)) .build(); if (query != null) { @@ -264,7 +268,7 @@ private Long getTotalRecords(Class entityClass, } /** - * Returns the total record count for a entity relationship + * Returns the total record count for a entity relationship. * @param relationship The relationship * @param filterExpression optional security and request filters * @param dictionary the entity dictionary diff --git a/elide-datastore/elide-datastore-inmemorydb/pom.xml b/elide-datastore/elide-datastore-inmemorydb/pom.xml index 0ba00ce907..6f75d107d5 100644 --- a/elide-datastore/elide-datastore-inmemorydb/pom.xml +++ b/elide-datastore/elide-datastore-inmemorydb/pom.xml @@ -14,7 +14,7 @@ com.yahoo.elide elide-datastore-parent-pom - 4.6.2-SNAPSHOT + 5.0.0-pr6-SNAPSHOT @@ -51,7 +51,7 @@ com.yahoo.elide elide-integration-tests - 4.6.2-SNAPSHOT + 5.0.0-pr6-SNAPSHOT test-jar test diff --git a/elide-datastore/elide-datastore-inmemorydb/src/test/java/com/yahoo/elide/datastores/inmemory/HashMapDataStoreTest.java b/elide-datastore/elide-datastore-inmemorydb/src/test/java/com/yahoo/elide/datastores/inmemory/HashMapDataStoreTest.java index a7dff3038b..d50da6b506 100644 --- a/elide-datastore/elide-datastore-inmemorydb/src/test/java/com/yahoo/elide/datastores/inmemory/HashMapDataStoreTest.java +++ b/elide-datastore/elide-datastore-inmemorydb/src/test/java/com/yahoo/elide/datastores/inmemory/HashMapDataStoreTest.java @@ -17,6 +17,7 @@ import com.yahoo.elide.example.beans.FirstBean; import com.yahoo.elide.example.beans.NonEntity; import com.yahoo.elide.example.beans.SecondBean; +import com.yahoo.elide.request.EntityProjection; import com.google.common.collect.ImmutableSet; import org.apache.commons.collections4.IterableUtils; @@ -25,7 +26,6 @@ import java.util.HashMap; import java.util.HashSet; -import java.util.Optional; import java.util.Set; /** @@ -33,10 +33,11 @@ */ public class HashMapDataStoreTest { private InMemoryDataStore inMemoryDataStore; + private EntityDictionary entityDictionary; @BeforeEach public void setup() { - final EntityDictionary entityDictionary = new EntityDictionary(new HashMap<>()); + entityDictionary = new EntityDictionary(new HashMap<>()); inMemoryDataStore = new InMemoryDataStore(FirstBean.class.getPackage()); inMemoryDataStore.populateEntityDictionary(entityDictionary); } @@ -56,13 +57,19 @@ public void testValidCommit() throws Exception { object.id = "0"; object.name = "Test"; try (DataStoreTransaction t = inMemoryDataStore.beginTransaction()) { - assertFalse(t.loadObjects(FirstBean.class, Optional.empty(), Optional.empty(), Optional.empty(), null).iterator().hasNext()); + assertFalse(t.loadObjects(EntityProjection.builder() + .type(FirstBean.class) + .build(), null).iterator().hasNext()); t.createObject(object, null); - assertFalse(t.loadObjects(FirstBean.class, Optional.empty(), Optional.empty(), Optional.empty(), null).iterator().hasNext()); + assertFalse(t.loadObjects(EntityProjection.builder() + .type(FirstBean.class) + .build(), null).iterator().hasNext()); t.commit(null); } try (DataStoreTransaction t = inMemoryDataStore.beginTransaction()) { - Iterable beans = t.loadObjects(FirstBean.class, Optional.empty(), Optional.empty(), Optional.empty(), null); + Iterable beans = t.loadObjects(EntityProjection.builder() + .type(FirstBean.class) + .build(), null); assertNotNull(beans); assertTrue(beans.iterator().hasNext()); FirstBean bean = (FirstBean) IterableUtils.first(beans); @@ -97,8 +104,9 @@ public void testCanGenerateIdsAfterDataCommitted() throws Exception { // and a meaningful ID is assigned Set names = new HashSet<>(); try (DataStoreTransaction t = inMemoryDataStore.beginTransaction()) { - for (Object objBean : t.loadObjects(FirstBean.class, - Optional.empty(), Optional.empty(), Optional.empty(), null)) { + for (Object objBean : t.loadObjects(EntityProjection.builder() + .type(FirstBean.class) + .build(), null)) { FirstBean bean = (FirstBean) objBean; names.add(bean.name); assertFalse(bean.id == null); diff --git a/elide-datastore/elide-datastore-jpa/pom.xml b/elide-datastore/elide-datastore-jpa/pom.xml index 9215bce1e0..a8a0da1cac 100644 --- a/elide-datastore/elide-datastore-jpa/pom.xml +++ b/elide-datastore/elide-datastore-jpa/pom.xml @@ -14,7 +14,7 @@ com.yahoo.elide elide-datastore-parent-pom - 4.6.2-SNAPSHOT + 5.0.0-pr6-SNAPSHOT @@ -52,7 +52,7 @@ com.yahoo.elide elide-datastore-hibernate - 4.6.2-SNAPSHOT + 5.0.0-pr6-SNAPSHOT @@ -75,7 +75,7 @@ com.yahoo.elide elide-integration-tests - 4.6.2-SNAPSHOT + 5.0.0-pr6-SNAPSHOT test-jar test diff --git a/elide-datastore/elide-datastore-jpa/src/main/java/com/yahoo/elide/datastores/jpa/transaction/AbstractJpaTransaction.java b/elide-datastore/elide-datastore-jpa/src/main/java/com/yahoo/elide/datastores/jpa/transaction/AbstractJpaTransaction.java index f6f4080c8e..36cf668f14 100644 --- a/elide-datastore/elide-datastore-jpa/src/main/java/com/yahoo/elide/datastores/jpa/transaction/AbstractJpaTransaction.java +++ b/elide-datastore/elide-datastore-jpa/src/main/java/com/yahoo/elide/datastores/jpa/transaction/AbstractJpaTransaction.java @@ -25,6 +25,8 @@ import com.yahoo.elide.datastores.jpa.porting.EntityManagerWrapper; import com.yahoo.elide.datastores.jpa.porting.QueryWrapper; import com.yahoo.elide.datastores.jpa.transaction.checker.PersistentCollectionChecker; +import com.yahoo.elide.request.EntityProjection; +import com.yahoo.elide.request.Relationship; import com.yahoo.elide.security.User; import lombok.extern.slf4j.Slf4j; @@ -136,17 +138,18 @@ public void createObject(Object entity, RequestScope scope) { /** * load a single record with id and filter. * - * @param entityClass class of query object + * @param projection the projection to query * @param id id of the query object - * @param filterExpression FilterExpression contains the predicates * @param scope Request scope associated with specific request */ @Override - public Object loadObject(Class entityClass, + public Object loadObject(EntityProjection projection, Serializable id, - Optional filterExpression, RequestScope scope) { + Class entityClass = projection.getType(); + FilterExpression filterExpression = projection.getFilterExpression(); + try { EntityDictionary dictionary = scope.getDictionary(); Class idType = dictionary.getIdType(entityClass); @@ -161,9 +164,9 @@ public Object loadObject(Class entityClass, idExpression = new FilterPredicate(idPath, Operator.FALSE, Collections.emptyList()); } - FilterExpression joinedExpression = filterExpression - .map(fe -> (FilterExpression) new AndFilterExpression(fe, idExpression)) - .orElse(idExpression); + FilterExpression joinedExpression = (filterExpression != null) + ? new AndFilterExpression(filterExpression, idExpression) + : idExpression; QueryWrapper query = (QueryWrapper) new RootCollectionFetchQueryBuilder(entityClass, dictionary, emWrapper) @@ -178,23 +181,24 @@ public Object loadObject(Class entityClass, @Override public Iterable loadObjects( - Class entityClass, - Optional filterExpression, - Optional sorting, - Optional pagination, + EntityProjection projection, RequestScope scope) { - pagination.ifPresent(p -> { - if (p.isGenerateTotals()) { - p.setPageTotals(getTotalRecords(entityClass, filterExpression, scope.getDictionary())); - } - }); + Class entityClass = projection.getType(); + Pagination pagination = projection.getPagination(); + FilterExpression filterExpression = projection.getFilterExpression(); + Sorting sorting = projection.getSorting(); + + if (pagination != null && pagination.isGenerateTotals()) { + pagination.setPageTotals(getTotalRecords(entityClass, + Optional.ofNullable(filterExpression), scope.getDictionary())); + } QueryWrapper query = (QueryWrapper) new RootCollectionFetchQueryBuilder(entityClass, scope.getDictionary(), emWrapper) - .withPossibleFilterExpression(filterExpression) - .withPossibleSorting(sorting) - .withPossiblePagination(pagination) + .withPossibleFilterExpression(Optional.ofNullable(filterExpression)) + .withPossibleSorting(Optional.ofNullable(sorting)) + .withPossiblePagination(Optional.ofNullable(pagination)) .build(); return query.getQuery().getResultList(); @@ -204,14 +208,15 @@ public Iterable loadObjects( public Object getRelation( DataStoreTransaction relationTx, Object entity, - String relationName, - Optional filterExpression, - Optional sorting, - Optional pagination, + Relationship relation, RequestScope scope) { + FilterExpression filterExpression = relation.getProjection().getFilterExpression(); + Sorting sorting = relation.getProjection().getSorting(); + Pagination pagination = relation.getProjection().getPagination(); + EntityDictionary dictionary = scope.getDictionary(); - Object val = com.yahoo.elide.core.PersistentResource.getValue(entity, relationName, scope); + Object val = com.yahoo.elide.core.PersistentResource.getValue(entity, relation.getName(), scope); if (val instanceof Collection) { Collection filteredVal = (Collection) val; if (IS_PERSISTENT_COLLECTION.test(filteredVal)) { @@ -220,31 +225,30 @@ public Object getRelation( * If there is no filtering or sorting required in the data store, and the pagination is default, * return the proxy and let the ORM manage the SQL generation. */ - if (! filterExpression.isPresent() && ! sorting.isPresent() - && (! pagination.isPresent() || (pagination.isPresent() && pagination.get().isDefaultInstance()))) { + if (filterExpression == null && sorting == null + && (pagination == null || (pagination.isDefaultInstance()))) { return val; } - Class relationClass = dictionary.getParameterizedType(entity, relationName); + Class relationClass = dictionary.getParameterizedType(entity, relation.getName()); RelationshipImpl relationship = new RelationshipImpl( dictionary.lookupEntityClass(entity.getClass()), relationClass, - relationName, + relation.getName(), entity, filteredVal); - pagination.ifPresent(p -> { - if (p.isGenerateTotals()) { - p.setPageTotals(getTotalRecords(relationship, filterExpression, dictionary)); - } - }); + if (pagination != null && pagination.isGenerateTotals()) { + pagination.setPageTotals(getTotalRecords(relationship, + Optional.ofNullable(filterExpression), scope.getDictionary())); + } QueryWrapper query = (QueryWrapper) new SubCollectionFetchQueryBuilder(relationship, dictionary, emWrapper) - .withPossibleFilterExpression(filterExpression) - .withPossibleSorting(sorting) - .withPossiblePagination(pagination) + .withPossibleFilterExpression(Optional.ofNullable(filterExpression)) + .withPossibleSorting(Optional.ofNullable(sorting)) + .withPossiblePagination(Optional.ofNullable(pagination)) .build(); if (query != null) { diff --git a/elide-datastore/elide-datastore-multiplex/pom.xml b/elide-datastore/elide-datastore-multiplex/pom.xml index b008f07ba3..b4a7c68677 100644 --- a/elide-datastore/elide-datastore-multiplex/pom.xml +++ b/elide-datastore/elide-datastore-multiplex/pom.xml @@ -10,7 +10,7 @@ com.yahoo.elide elide-datastore-parent-pom - 4.6.2-SNAPSHOT + 5.0.0-pr6-SNAPSHOT @@ -47,19 +47,19 @@ com.yahoo.elide elide-datastore-inmemorydb - 4.6.2-SNAPSHOT + 5.0.0-pr6-SNAPSHOT test com.yahoo.elide elide-datastore-hibernate5 - 4.6.2-SNAPSHOT + 5.0.0-pr6-SNAPSHOT test com.yahoo.elide elide-integration-tests - 4.6.2-SNAPSHOT + 5.0.0-pr6-SNAPSHOT test-jar test diff --git a/elide-datastore/elide-datastore-multiplex/src/main/java/com/yahoo/elide/datastores/multiplex/MultiplexManager.java b/elide-datastore/elide-datastore-multiplex/src/main/java/com/yahoo/elide/datastores/multiplex/MultiplexManager.java index 322d734247..f9f39803d8 100644 --- a/elide-datastore/elide-datastore-multiplex/src/main/java/com/yahoo/elide/datastores/multiplex/MultiplexManager.java +++ b/elide-datastore/elide-datastore-multiplex/src/main/java/com/yahoo/elide/datastores/multiplex/MultiplexManager.java @@ -9,6 +9,9 @@ import com.yahoo.elide.core.DataStoreTransaction; import com.yahoo.elide.core.EntityDictionary; +import lombok.AccessLevel; +import lombok.Setter; + import java.util.Arrays; import java.util.List; import java.util.concurrent.ConcurrentHashMap; @@ -28,10 +31,12 @@ *
  • Attempt to reverse DB1 commit fails * */ -public class MultiplexManager implements DataStore { +public final class MultiplexManager implements DataStore { protected final List dataStores; protected final ConcurrentHashMap, DataStore> dataStoreMap = new ConcurrentHashMap<>(); + + @Setter(AccessLevel.PROTECTED) private EntityDictionary dictionary; /** @@ -55,7 +60,13 @@ public void populateEntityDictionary(EntityDictionary dictionary) { this.dataStoreMap.put(cls, dataStore); // bind to multiplex dictionary dictionary.bindEntity(cls); - dictionary.bindInitializer(subordinateDictionary::initializeEntity, cls); + // copy attribute arguments + subordinateDictionary.getAttributes(cls).forEach( + attribute -> dictionary.addArgumentsToAttribute( + cls, + attribute, + subordinateDictionary.getAttributeArguments(cls, attribute)) + ); } } } diff --git a/elide-datastore/elide-datastore-multiplex/src/main/java/com/yahoo/elide/datastores/multiplex/MultiplexTransaction.java b/elide-datastore/elide-datastore-multiplex/src/main/java/com/yahoo/elide/datastores/multiplex/MultiplexTransaction.java index 6efb11fbad..b6685a055b 100644 --- a/elide-datastore/elide-datastore-multiplex/src/main/java/com/yahoo/elide/datastores/multiplex/MultiplexTransaction.java +++ b/elide-datastore/elide-datastore-multiplex/src/main/java/com/yahoo/elide/datastores/multiplex/MultiplexTransaction.java @@ -17,6 +17,9 @@ import com.yahoo.elide.core.filter.expression.PredicateExtractionVisitor; import com.yahoo.elide.core.pagination.Pagination; import com.yahoo.elide.core.sort.Sorting; +import com.yahoo.elide.request.Attribute; +import com.yahoo.elide.request.EntityProjection; +import com.yahoo.elide.request.Relationship; import com.yahoo.elide.security.User; import java.io.IOException; @@ -69,27 +72,18 @@ public void createObject(Object entity, RequestScope scope) { getTransaction(entity).createObject(entity, scope); } - @Override - public Object loadObject(Class entityClass, + public Object loadObject(EntityProjection projection, Serializable id, - Optional filterExpression, RequestScope scope) { - return getTransaction(entityClass).loadObject(entityClass, id, filterExpression, scope); + return getTransaction(projection.getType()).loadObject(projection, id, scope); } @Override public Iterable loadObjects( - Class entityClass, - Optional filterExpression, - Optional sorting, - Optional pagination, + EntityProjection projection, RequestScope scope) { - return getTransaction(entityClass).loadObjects(entityClass, - filterExpression, - sorting, - pagination, - scope); + return getTransaction(projection.getType()).loadObjects(projection, scope); } @Override @@ -153,36 +147,44 @@ protected DataStoreTransaction getRelationTransaction(Object object, String rela @Override public Object getRelation(DataStoreTransaction relationTx, Object entity, - String relationName, - Optional filter, - Optional sorting, - Optional pagination, + Relationship relation, RequestScope scope) { - relationTx = getRelationTransaction(entity, relationName); + + FilterExpression filter = relation.getProjection().getFilterExpression(); + Sorting sorting = relation.getProjection().getSorting(); + Pagination pagination = relation.getProjection().getPagination(); + + relationTx = getRelationTransaction(entity, relation.getName()); DataStoreTransaction entityTransaction = getTransaction(entity.getClass()); EntityDictionary dictionary = scope.getDictionary(); - Class relationClass = dictionary.getParameterizedType(entity, relationName); + Class relationClass = dictionary.getParameterizedType(entity, relation.getName()); String idFieldName = dictionary.getIdFieldName(relationClass); // If different transactions, check if bridgeable and try to bridge if (entityTransaction != relationTx && relationTx instanceof BridgeableTransaction) { BridgeableTransaction bridgeableTx = (BridgeableTransaction) relationTx; - RelationshipType relationType = dictionary.getRelationshipType(entity.getClass(), relationName); - Serializable id = filter.map(fe -> extractId(fe, idFieldName, relationClass)).orElse(null); + RelationshipType relationType = dictionary.getRelationshipType(entity.getClass(), relation.getName()); + Serializable id = (filter != null) ? extractId(filter, idFieldName, relationClass) : null; if (relationType.isToMany()) { return id == null ? bridgeableTx.bridgeableLoadObjects( - this, entity, relationName, filter, sorting, pagination, scope) - : bridgeableTx.bridgeableLoadObject(this, entity, relationName, id, filter, scope); + this, entity, relation.getName(), + Optional.ofNullable(filter), + Optional.ofNullable(relation.getProjection().getSorting()), + Optional.ofNullable(relation.getProjection().getPagination()), + scope) + : bridgeableTx.bridgeableLoadObject(this, entity, relation.getName(), + id, Optional.ofNullable(filter), scope); } - return bridgeableTx.bridgeableLoadObject(this, entity, relationName, id, filter, scope); + return bridgeableTx.bridgeableLoadObject(this, entity, relation.getName(), id, + Optional.ofNullable(filter), scope); } // Otherwise, rely on existing underlying transaction to call correctly into relationTx - return entityTransaction.getRelation(relationTx, entity, relationName, filter, sorting, pagination, scope); + return entityTransaction.getRelation(relationTx, entity, relation, scope); } @Override @@ -206,16 +208,15 @@ public void updateToOneRelation(DataStoreTransaction relationTx, Object entity, } @Override - public Object getAttribute(Object entity, - String attributeName, RequestScope scope) { + public Object getAttribute(Object entity, Attribute attribute, RequestScope scope) { DataStoreTransaction transaction = getTransaction(entity.getClass()); - return transaction.getAttribute(entity, attributeName, scope); + return transaction.getAttribute(entity, attribute, scope); } @Override - public void setAttribute(Object entity, String attributeName, Object attributeValue, RequestScope scope) { + public void setAttribute(Object entity, Attribute attribute, RequestScope scope) { DataStoreTransaction transaction = getTransaction(entity.getClass()); - transaction.setAttribute(entity, attributeName, attributeValue, scope); + transaction.setAttribute(entity, attribute, scope); } @Override diff --git a/elide-datastore/elide-datastore-multiplex/src/main/java/com/yahoo/elide/datastores/multiplex/MultiplexWriteTransaction.java b/elide-datastore/elide-datastore-multiplex/src/main/java/com/yahoo/elide/datastores/multiplex/MultiplexWriteTransaction.java index d20e03097d..5f70e1bf02 100644 --- a/elide-datastore/elide-datastore-multiplex/src/main/java/com/yahoo/elide/datastores/multiplex/MultiplexWriteTransaction.java +++ b/elide-datastore/elide-datastore-multiplex/src/main/java/com/yahoo/elide/datastores/multiplex/MultiplexWriteTransaction.java @@ -10,9 +10,8 @@ import com.yahoo.elide.core.RequestScope; import com.yahoo.elide.core.exceptions.HttpStatusException; import com.yahoo.elide.core.exceptions.TransactionException; -import com.yahoo.elide.core.filter.expression.FilterExpression; -import com.yahoo.elide.core.pagination.Pagination; -import com.yahoo.elide.core.sort.Sorting; +import com.yahoo.elide.request.EntityProjection; +import com.yahoo.elide.request.Relationship; import java.io.IOException; import java.io.Serializable; @@ -23,7 +22,6 @@ import java.util.IdentityHashMap; import java.util.List; import java.util.Map.Entry; -import java.util.Optional; import javax.ws.rs.WebApplicationException; import javax.ws.rs.core.MultivaluedHashMap; @@ -170,36 +168,28 @@ private Object cloneObject(Object object) { } @Override - public Object loadObject(Class entityClass, + public Object loadObject(EntityProjection projection, Serializable id, - Optional filterExpression, RequestScope scope) { - DataStoreTransaction transaction = getTransaction(entityClass); - return hold(transaction, transaction.loadObject(entityClass, id, filterExpression, scope)); + DataStoreTransaction transaction = getTransaction(projection.getType()); + return hold(transaction, transaction.loadObject(projection, id, scope)); } @Override public Iterable loadObjects( - Class entityClass, - Optional filterExpression, - Optional sorting, - Optional pagination, + EntityProjection projection, RequestScope scope) { - DataStoreTransaction transaction = getTransaction(entityClass); - return hold(transaction, transaction.loadObjects(entityClass, filterExpression, sorting, pagination, scope)); + DataStoreTransaction transaction = getTransaction(projection.getType()); + return hold(transaction, transaction.loadObjects(projection, scope)); } @Override public Object getRelation(DataStoreTransaction relationTx, Object entity, - String relationName, - Optional filter, - Optional sorting, - Optional pagination, + Relationship relationship, RequestScope scope) { DataStoreTransaction transaction = getTransaction(entity.getClass()); - Object relation = super.getRelation(relationTx, entity, relationName, - filter, sorting, pagination, scope); + Object relation = super.getRelation(relationTx, entity, relationship, scope); if (relation instanceof Iterable) { return hold(transaction, (Iterable) relation); diff --git a/elide-datastore/elide-datastore-multiplex/src/test/java/com/yahoo/elide/datastores/multiplex/MultiplexManagerTest.java b/elide-datastore/elide-datastore-multiplex/src/test/java/com/yahoo/elide/datastores/multiplex/MultiplexManagerTest.java index 33526c0b50..3b1d054fda 100644 --- a/elide-datastore/elide-datastore-multiplex/src/test/java/com/yahoo/elide/datastores/multiplex/MultiplexManagerTest.java +++ b/elide-datastore/elide-datastore-multiplex/src/test/java/com/yahoo/elide/datastores/multiplex/MultiplexManagerTest.java @@ -18,6 +18,7 @@ import com.yahoo.elide.datastores.inmemory.InMemoryDataStore; import com.yahoo.elide.example.beans.FirstBean; import com.yahoo.elide.example.other.OtherBean; +import com.yahoo.elide.request.EntityProjection; import com.google.common.collect.Lists; import org.apache.commons.collections4.IterableUtils; @@ -30,7 +31,6 @@ import java.io.IOException; import java.util.ArrayList; import java.util.HashMap; -import java.util.Optional; /** * MultiplexManager tests. @@ -39,10 +39,11 @@ @TestInstance(TestInstance.Lifecycle.PER_CLASS) public class MultiplexManagerTest { private MultiplexManager multiplexManager; + private EntityDictionary entityDictionary; @BeforeAll public void setup() { - final EntityDictionary entityDictionary = new EntityDictionary(new HashMap<>()); + entityDictionary = new EntityDictionary(new HashMap<>()); final InMemoryDataStore inMemoryDataStore1 = new InMemoryDataStore(FirstBean.class.getPackage()); final InMemoryDataStore inMemoryDataStore2 = new InMemoryDataStore(OtherBean.class.getPackage()); multiplexManager = new MultiplexManager(inMemoryDataStore1, inMemoryDataStore2); @@ -62,15 +63,21 @@ public void testValidCommit() throws IOException { object.id = null; object.name = "Test"; try (DataStoreTransaction t = multiplexManager.beginTransaction()) { - assertFalse(t.loadObjects(FirstBean.class, Optional.empty(), Optional.empty(), Optional.empty(), null) + assertFalse(t.loadObjects(EntityProjection.builder() + .type(FirstBean.class) + .build(), null) .iterator().hasNext()); t.createObject(object, null); - assertFalse(t.loadObjects(FirstBean.class, Optional.empty(), Optional.empty(), Optional.empty(), null) + assertFalse(t.loadObjects(EntityProjection.builder() + .type(FirstBean.class) + .build(), null) .iterator().hasNext()); t.commit(null); } try (DataStoreTransaction t = multiplexManager.beginTransaction()) { - Iterable beans = t.loadObjects(FirstBean.class, Optional.empty(), Optional.empty(), Optional.empty(), null); + Iterable beans = t.loadObjects(EntityProjection.builder() + .type(FirstBean.class) + .build(), null); assertNotNull(beans); assertTrue(beans.iterator().hasNext()); FirstBean bean = (FirstBean) IterableUtils.first(beans); @@ -90,19 +97,25 @@ public void partialCommitFailure() throws IOException { assertEquals(ds2, multiplexManager.getSubManager(OtherBean.class)); try (DataStoreTransaction t = ds1.beginTransaction()) { - assertFalse(t.loadObjects(FirstBean.class, Optional.empty(), Optional.empty(), Optional.empty(), null).iterator().hasNext()); + assertFalse(t.loadObjects(EntityProjection.builder() + .type(FirstBean.class) + .build(), null).iterator().hasNext()); FirstBean firstBean = FirstBean.class.newInstance(); firstBean.name = "name"; t.createObject(firstBean, null); //t.save(firstBean); - assertFalse(t.loadObjects(FirstBean.class, Optional.empty(), Optional.empty(), Optional.empty(), null).iterator().hasNext()); + assertFalse(t.loadObjects(EntityProjection.builder() + .type(FirstBean.class) + .build(), null).iterator().hasNext()); t.commit(null); } catch (InstantiationException | IllegalAccessException e) { log.error("", e); } try (DataStoreTransaction t = multiplexManager.beginTransaction()) { - FirstBean firstBean = (FirstBean) t.loadObjects(FirstBean.class, Optional.empty(), Optional.empty(), Optional.empty(), null).iterator().next(); + FirstBean firstBean = (FirstBean) t.loadObjects(EntityProjection.builder() + .type(FirstBean.class) + .build(), null).iterator().next(); firstBean.name = "update"; t.save(firstBean, null); OtherBean otherBean = OtherBean.class.newInstance(); @@ -119,7 +132,9 @@ public void partialCommitFailure() throws IOException { } // verify state try (DataStoreTransaction t = ds1.beginTransaction()) { - Iterable beans = t.loadObjects(FirstBean.class, Optional.empty(), Optional.empty(), Optional.empty(), null); + Iterable beans = t.loadObjects(EntityProjection.builder() + .type(FirstBean.class) + .build(), null); assertNotNull(beans); ArrayList list = Lists.newArrayList(beans.iterator()); assertEquals(list.size(), 1); diff --git a/elide-datastore/elide-datastore-multiplex/src/test/java/com/yahoo/elide/datastores/multiplex/TestDataStore.java b/elide-datastore/elide-datastore-multiplex/src/test/java/com/yahoo/elide/datastores/multiplex/TestDataStore.java index e1d2414a84..1712965e8a 100644 --- a/elide-datastore/elide-datastore-multiplex/src/test/java/com/yahoo/elide/datastores/multiplex/TestDataStore.java +++ b/elide-datastore/elide-datastore-multiplex/src/test/java/com/yahoo/elide/datastores/multiplex/TestDataStore.java @@ -10,14 +10,11 @@ import com.yahoo.elide.core.EntityDictionary; import com.yahoo.elide.core.RequestScope; import com.yahoo.elide.core.exceptions.TransactionException; -import com.yahoo.elide.core.filter.expression.FilterExpression; -import com.yahoo.elide.core.pagination.Pagination; -import com.yahoo.elide.core.sort.Sorting; +import com.yahoo.elide.request.EntityProjection; import com.yahoo.elide.utils.ClassScanner; import java.io.IOException; import java.io.Serializable; -import java.util.Optional; import javax.persistence.Entity; @@ -68,19 +65,15 @@ public void createObject(Object entity, RequestScope scope) { } @Override - public Object loadObject(Class entityClass, - Serializable id, - Optional filterExpression, - RequestScope scope) { + public Object loadObject(EntityProjection projection, + Serializable id, + RequestScope scope) { throw new TransactionException(null); } @Override public Iterable loadObjects( - Class entityClass, - Optional filterExpression, - Optional sorting, - Optional pagination, + EntityProjection projection, RequestScope scope) { throw new TransactionException(null); } diff --git a/elide-datastore/elide-datastore-multiplex/src/test/java/com/yahoo/elide/datastores/multiplex/bridgeable/BridgeableRedisStore.java b/elide-datastore/elide-datastore-multiplex/src/test/java/com/yahoo/elide/datastores/multiplex/bridgeable/BridgeableRedisStore.java index 19d6335329..5cb16dc6e7 100644 --- a/elide-datastore/elide-datastore-multiplex/src/test/java/com/yahoo/elide/datastores/multiplex/bridgeable/BridgeableRedisStore.java +++ b/elide-datastore/elide-datastore-multiplex/src/test/java/com/yahoo/elide/datastores/multiplex/bridgeable/BridgeableRedisStore.java @@ -9,7 +9,6 @@ import com.yahoo.elide.core.DataStoreTransaction; import com.yahoo.elide.core.EntityDictionary; import com.yahoo.elide.core.Path; -import com.yahoo.elide.core.PersistentResource; import com.yahoo.elide.core.RequestScope; import com.yahoo.elide.core.filter.FilterPredicate; import com.yahoo.elide.core.filter.InPredicate; @@ -26,6 +25,8 @@ import com.yahoo.elide.datastores.multiplex.MultiplexTransaction; import com.yahoo.elide.example.beans.HibernateUser; import com.yahoo.elide.example.hbase.beans.RedisActions; +import com.yahoo.elide.request.EntityProjection; +import com.yahoo.elide.request.Relationship; import com.yahoo.elide.security.User; import lombok.AllArgsConstructor; @@ -61,19 +62,18 @@ public DataStoreTransaction beginReadTransaction() { public class ExampleRedisTransaction implements DataStoreTransaction, BridgeableTransaction { @Override - public Object loadObject(Class entityClass, + public Object loadObject(EntityProjection projection, Serializable id, - Optional filterExpression, RequestScope scope) { - if (entityClass != RedisActions.class) { - log.debug("Tried to load unexpected object from redis: {}", entityClass); + if (projection.getType() != RedisActions.class) { + log.debug("Tried to load unexpected object from redis: {}", projection.getType()); throw new RuntimeException("Tried to load unexpected object from redis!"); } String key = RedisActions.class.getCanonicalName(); - if (filterExpression.isPresent()) { - FilterExpression fe = filterExpression.get(); + FilterExpression fe = projection.getFilterExpression(); + if (fe != null) { RedisFilter filter = fe.accept(new FilterExpressionParser()); if ("user_id".equals(filter.getFieldName())) { Iterable values = fetchValues(key, @@ -89,26 +89,23 @@ public Object loadObject(Class entityClass, } @Override - public Iterable loadObjects(Class entityClass, - Optional filterExpression, - Optional sorting, - Optional pagination, + public Iterable loadObjects(EntityProjection projection, RequestScope scope) { - if (entityClass != RedisActions.class) { - log.debug("Tried to load unexpected object from redis: {}", entityClass); + if (projection.getType() != RedisActions.class) { + log.debug("Tried to load unexpected object from redis: {}", projection.getType()); throw new RuntimeException("Tried to load unexpected object from redis!"); } String key = RedisActions.class.getCanonicalName(); - return filterExpression + return Optional.ofNullable(projection.getFilterExpression()) .map(fe -> { RedisFilter filter = fe.accept(new FilterExpressionParser()); if ("user_id".equals(filter.getFieldName())) { return fetchValues(key, v -> v.startsWith("user" + filter.getValues().get(0) + ":")); } - log.error("Received bad filter: {} for type: {}", filter, entityClass); + log.error("Received bad filter: {} for type: {}", filter, projection.getType()); throw new UnsupportedOperationException("Cannot filter object of that type"); }) .orElseGet(() -> fetchValues(key, unused -> true)); @@ -133,11 +130,6 @@ private RedisActions deserializeAction(Map.Entry entry) { return action; } - @Override - public Object getAttribute(Object entity, String attributeName, RequestScope scope) { - return PersistentResource.getValue(entity, attributeName, scope); - } - // ---- Bridgeable Interfaces ---- @Override @@ -147,9 +139,9 @@ public Object bridgeableLoadObject(MultiplexTransaction muxTx, Object parent, St Class entityClass = dictionary.getParameterizedType(parent, relationName); HibernateUser user = (HibernateUser) parent; if ("specialAction".equals(relationName)) { - return muxTx.loadObject(entityClass, + return muxTx.loadObject( + EntityProjection.builder().type(entityClass).build(), String.valueOf(user.getSpecialActionId()), - Optional.empty(), scope); } else if ("redisActions".equals(relationName)) { FilterExpression updatedExpression = new InPredicate( @@ -157,9 +149,11 @@ public Object bridgeableLoadObject(MultiplexTransaction muxTx, Object parent, St String.valueOf(((HibernateUser) parent).getId()) ); - return muxTx.loadObject(entityClass, + return muxTx.loadObject(EntityProjection.builder() + .type(entityClass) + .filterExpression(updatedExpression) + .build(), String.valueOf(lookupId), - Optional.of(updatedExpression), scope); } } @@ -176,11 +170,12 @@ public Iterable bridgeableLoadObjects(MultiplexTransaction muxTx, Object new Path.PathElement(entityClass, String.class, "user_id"), String.valueOf(((HibernateUser) parent).getId()) ); - return muxTx.loadObjects(entityClass, - Optional.of(filterExpression), - sorting, - pagination, - scope); + return muxTx.loadObjects(EntityProjection.builder() + .type(entityClass) + .filterExpression(filterExpression) + .sorting(sorting.orElse(null)) + .pagination(pagination.orElse(null)) + .build(), scope); } log.error("Tried to bridge from parent: {} to relation name: {}", parent, relationName); throw new RuntimeException("Unsupported bridging attempted!"); @@ -190,11 +185,7 @@ public Iterable bridgeableLoadObjects(MultiplexTransaction muxTx, Object @Override public Object getRelation(DataStoreTransaction relationTx, - Object entity, - String relationName, - Optional filterExpression, - Optional sorting, - Optional pagination, RequestScope scope) { + Object entity, Relationship relationship, RequestScope scope) { throw new UnsupportedOperationException("No redis relationships currently supported."); } @@ -208,11 +199,6 @@ public void updateToOneRelation(DataStoreTransaction relationTx, Object entity, } - @Override - public void setAttribute(Object entity, String attributeName, Object attributeValue, RequestScope scope) { - - } - @Override public void close() throws IOException { diff --git a/elide-datastore/elide-datastore-noop/pom.xml b/elide-datastore/elide-datastore-noop/pom.xml index 45fcf19e58..f51f1d9618 100644 --- a/elide-datastore/elide-datastore-noop/pom.xml +++ b/elide-datastore/elide-datastore-noop/pom.xml @@ -6,7 +6,7 @@ com.yahoo.elide elide-datastore-parent-pom - 4.6.2-SNAPSHOT + 5.0.0-pr6-SNAPSHOT diff --git a/elide-datastore/elide-datastore-noop/src/main/java/com/yahoo/elide/datastores/noop/NoopTransaction.java b/elide-datastore/elide-datastore-noop/src/main/java/com/yahoo/elide/datastores/noop/NoopTransaction.java index 3bb0862a96..8e51831449 100644 --- a/elide-datastore/elide-datastore-noop/src/main/java/com/yahoo/elide/datastores/noop/NoopTransaction.java +++ b/elide-datastore/elide-datastore-noop/src/main/java/com/yahoo/elide/datastores/noop/NoopTransaction.java @@ -8,16 +8,13 @@ import com.yahoo.elide.core.DataStoreTransaction; import com.yahoo.elide.core.PersistentResource; import com.yahoo.elide.core.RequestScope; -import com.yahoo.elide.core.filter.expression.FilterExpression; -import com.yahoo.elide.core.pagination.Pagination; -import com.yahoo.elide.core.sort.Sorting; +import com.yahoo.elide.request.EntityProjection; import lombok.extern.slf4j.Slf4j; import java.io.IOException; import java.io.Serializable; import java.util.Collections; -import java.util.Optional; /** * Noop transaction. Specifically, this transaction does not perform any actions (i.e. no operation). @@ -74,24 +71,22 @@ public void createObject(Object entity, RequestScope scope) { /** * No-op transaction, do nothing. - * @param entityClass the type of class to load + * @param projection the projection to query * @param id - the ID of the object to load. - * @param filterExpression - security filters that can be evaluated in the data store. * @param scope - the current request scope. It is optional for the data store to attempt evaluation. * @return a new persistent resource with a new instance of {@code entityClass} */ @Override - public Object loadObject(Class entityClass, + public Object loadObject(EntityProjection projection, Serializable id, - Optional filterExpression, RequestScope scope) { // Loads are supported but empty object (with specified id) is returned. // NOTE: This is primarily useful for enabling objects of solely computed properties to be fetched. Object entity; try { - entity = entityClass.newInstance(); + entity = projection.getType().newInstance(); } catch (IllegalAccessException | InstantiationException e) { - log.error("Could not load object {} through NoopStore", entityClass, e); + log.error("Could not load object {} through NoopStore", projection.getType(), e); throw new RuntimeException(e); } @@ -105,22 +100,15 @@ public Object loadObject(Class entityClass, /** * No-op transaction, do nothing. - * @param entityClass - the class to load - * @param filterExpression - filters that can be evaluated in the data store. - * It is optional for the data store to attempt evaluation. - * @param sorting - sorting which can be pushed down to the data store. - * @param pagination - pagination which can be pushed down to the data store. + * @param projection - the projection to load * @param scope - contains request level metadata. * @return a {@link Collections#singletonList} with a new persistent resource with id 1 */ @Override - public Iterable loadObjects(Class entityClass, - Optional filterExpression, - Optional sorting, - Optional pagination, + public Iterable loadObjects(EntityProjection projection, RequestScope scope) { // Default behavior: load object 1 and return as an array - return Collections.singletonList(this.loadObject(entityClass, 1L, filterExpression, scope)); + return Collections.singletonList(this.loadObject(projection, 1L, scope)); } /** diff --git a/elide-datastore/elide-datastore-noop/src/test/java/com/yahoo/elide/beans/NoopBean.java b/elide-datastore/elide-datastore-noop/src/test/java/com/yahoo/elide/beans/NoopBean.java index 024b6a6a3c..2baaf27ca3 100644 --- a/elide-datastore/elide-datastore-noop/src/test/java/com/yahoo/elide/beans/NoopBean.java +++ b/elide-datastore/elide-datastore-noop/src/test/java/com/yahoo/elide/beans/NoopBean.java @@ -10,7 +10,7 @@ import javax.persistence.Id; /** - * Simple bean intended to not be persisted + * Simple bean intended to not be persisted. */ @Include(type = "theNoopBean") public class NoopBean { diff --git a/elide-datastore/elide-datastore-noop/src/test/java/com/yahoo/elide/datastores/noop/NoopTransactionTest.java b/elide-datastore/elide-datastore-noop/src/test/java/com/yahoo/elide/datastores/noop/NoopTransactionTest.java index 67d1f43f95..26aed1626f 100644 --- a/elide-datastore/elide-datastore-noop/src/test/java/com/yahoo/elide/datastores/noop/NoopTransactionTest.java +++ b/elide-datastore/elide-datastore-noop/src/test/java/com/yahoo/elide/datastores/noop/NoopTransactionTest.java @@ -15,6 +15,7 @@ import com.yahoo.elide.core.ObjectEntityCache; import com.yahoo.elide.core.RequestScope; import com.yahoo.elide.jsonapi.JsonApiMapper; +import com.yahoo.elide.request.EntityProjection; import com.fasterxml.jackson.databind.ObjectMapper; import org.apache.commons.collections4.IterableUtils; @@ -23,17 +24,17 @@ import org.junit.jupiter.api.TestInstance; import java.util.HashMap; -import java.util.Optional; @TestInstance(TestInstance.Lifecycle.PER_CLASS) public class NoopTransactionTest { - DataStoreTransaction tx = new NoopTransaction(); - NoopBean bean = new NoopBean(); - RequestScope requestScope; + private DataStoreTransaction tx = new NoopTransaction(); + private NoopBean bean = new NoopBean(); + private RequestScope requestScope; + private EntityDictionary dictionary; @BeforeAll public void setup() { - EntityDictionary dictionary = new EntityDictionary(new HashMap<>()); + dictionary = new EntityDictionary(new HashMap<>()); dictionary.bindEntity(NoopBean.class); requestScope = mock(RequestScope.class); JsonApiMapper mapper = mock(JsonApiMapper.class); @@ -79,13 +80,17 @@ public void testCreateObject() throws Exception { public void testLoadObject() throws Exception { // Should return bean with id set - NoopBean bean = (NoopBean) tx.loadObject(NoopBean.class, 1, Optional.empty(), requestScope); - assertEquals((Long) 1L, bean.getId()); + NoopBean bean = (NoopBean) tx.loadObject(EntityProjection.builder() + .type(NoopBean.class) + .build(), 1, requestScope); + assertEquals(bean.getId(), (Long) 1L); } @Test public void testLoadObjects() throws Exception { - Iterable iterable = (Iterable) tx.loadObjects(NoopBean.class, Optional.empty(), Optional.empty(), Optional.empty(), requestScope); + Iterable iterable = (Iterable) tx.loadObjects(EntityProjection.builder() + .type(NoopBean.class) + .build(), requestScope); NoopBean bean = IterableUtils.first(iterable); assertEquals((Long) 1L, bean.getId()); } diff --git a/elide-datastore/elide-datastore-search/pom.xml b/elide-datastore/elide-datastore-search/pom.xml index 3396c43f60..1523f0fd57 100644 --- a/elide-datastore/elide-datastore-search/pom.xml +++ b/elide-datastore/elide-datastore-search/pom.xml @@ -12,7 +12,7 @@ com.yahoo.elide elide-datastore-parent-pom - 4.6.2-SNAPSHOT + 5.0.0-pr6-SNAPSHOT @@ -57,7 +57,7 @@ com.yahoo.elide elide-integration-tests - 4.6.2-SNAPSHOT + 5.0.0-pr6-SNAPSHOT test-jar test @@ -65,7 +65,7 @@ com.yahoo.elide elide-datastore-jpa - 4.6.2-SNAPSHOT + 5.0.0-pr6-SNAPSHOT test diff --git a/elide-datastore/elide-datastore-search/src/main/java/com/yahoo/elide/datastores/search/SearchDataTransaction.java b/elide-datastore/elide-datastore-search/src/main/java/com/yahoo/elide/datastores/search/SearchDataTransaction.java index dc69d282c9..5a0dcfb7f0 100644 --- a/elide-datastore/elide-datastore-search/src/main/java/com/yahoo/elide/datastores/search/SearchDataTransaction.java +++ b/elide-datastore/elide-datastore-search/src/main/java/com/yahoo/elide/datastores/search/SearchDataTransaction.java @@ -23,6 +23,7 @@ import com.yahoo.elide.core.filter.expression.PredicateExtractionVisitor; import com.yahoo.elide.core.pagination.Pagination; import com.yahoo.elide.core.sort.Sorting; +import com.yahoo.elide.request.EntityProjection; import org.apache.lucene.search.Query; import org.apache.lucene.search.Sort; @@ -69,26 +70,25 @@ public SearchDataTransaction(DataStoreTransaction tx, } @Override - public Iterable loadObjects(Class entityClass, - Optional filterExpression, - Optional sorting, - Optional pagination, + public Iterable loadObjects(EntityProjection projection, RequestScope requestScope) { - if (!filterExpression.isPresent()) { - return super.loadObjects(entityClass, filterExpression, sorting, pagination, requestScope); + if (projection.getFilterExpression() == null) { + return super.loadObjects(projection, requestScope); } - boolean canSearch = (canSearch(entityClass, filterExpression.get()) != NONE); + boolean canSearch = (canSearch(projection.getType(), projection.getFilterExpression()) != NONE); - if (mustSort(sorting, entityClass)) { - canSearch = canSearch && canSort(sorting.get(), entityClass); + if (mustSort(Optional.ofNullable(projection.getSorting()), projection.getType())) { + canSearch = canSearch && canSort(projection.getSorting(), projection.getType()); } if (canSearch) { - return search(entityClass, filterExpression.get(), sorting, pagination); + return search(projection.getType(), projection.getFilterExpression(), + Optional.ofNullable(projection.getSorting()), + Optional.ofNullable(projection.getPagination())); } - return super.loadObjects(entityClass, filterExpression, sorting, pagination, requestScope); + return super.loadObjects(projection, requestScope); } /** diff --git a/elide-datastore/elide-datastore-search/src/test/java/com/yahoo/elide/datastores/search/DataStoreLoadTest.java b/elide-datastore/elide-datastore-search/src/test/java/com/yahoo/elide/datastores/search/DataStoreLoadTest.java index 8e30ae5e5f..0d22fb5bf9 100644 --- a/elide-datastore/elide-datastore-search/src/test/java/com/yahoo/elide/datastores/search/DataStoreLoadTest.java +++ b/elide-datastore/elide-datastore-search/src/test/java/com/yahoo/elide/datastores/search/DataStoreLoadTest.java @@ -25,6 +25,7 @@ import com.yahoo.elide.core.pagination.Pagination; import com.yahoo.elide.core.sort.Sorting; import com.yahoo.elide.datastores.search.models.Item; +import com.yahoo.elide.request.EntityProjection; import com.yahoo.elide.utils.coerce.CoerceUtil; import com.yahoo.elide.utils.coerce.converters.ISO8601DateSerde; @@ -40,7 +41,6 @@ import java.util.HashMap; import java.util.List; import java.util.Map; -import java.util.Optional; import java.util.stream.Collectors; import java.util.stream.StreamSupport; @@ -54,9 +54,10 @@ public class DataStoreLoadTest { private SearchDataStore searchStore; private DataStoreTransaction wrappedTransaction; private RequestScope mockScope; + private EntityDictionary dictionary; public DataStoreLoadTest() { - EntityDictionary dictionary = new EntityDictionary(new HashMap<>()); + dictionary = new EntityDictionary(new HashMap<>()); dictionary.bindEntity(Item.class); filterParser = new RSQLFilterDialect(dictionary); @@ -99,12 +100,16 @@ public void testEqualityPredicate() throws Exception { //Case sensitive query against case insensitive index must lowercase FilterExpression filter = filterParser.parseFilterExpression("name==drum", Item.class, false); - Iterable loaded = testTransaction.loadObjects(Item.class, Optional.of(filter), Optional.empty(), Optional.empty(), mockScope); + Iterable loaded = testTransaction.loadObjects( + EntityProjection.builder() + .type(Item.class) + .filterExpression(filter) + .build(), mockScope); assertListContains(loaded, Lists.newArrayList()); /* This query should hit the underlying store */ - verify(wrappedTransaction, times(1)).loadObjects(any(), any(), any(), any(), any()); + verify(wrappedTransaction, times(1)).loadObjects(any(), any()); } @Test @@ -115,10 +120,13 @@ public void testEscapedPrefixPredicate() throws Exception { /* Verify that '-' is escaped before we run the query */ FilterExpression filter = filterParser.parseFilterExpression("name==-lucen*", Item.class, false); - Iterable loaded = testTransaction.loadObjects(Item.class, Optional.of(filter), Optional.empty(), Optional.empty(), mockScope); + Iterable loaded = testTransaction.loadObjects(EntityProjection.builder() + .type(Item.class) + .filterExpression(filter) + .build(), mockScope); assertListContains(loaded, Lists.newArrayList(6L)); - verify(wrappedTransaction, never()).loadObjects(any(), any(), any(), any(), any()); + verify(wrappedTransaction, never()).loadObjects(any(), any()); } @Test @@ -127,10 +135,13 @@ public void testEmptyResult() throws Exception { FilterExpression filter = filterParser.parseFilterExpression("name==+lucen*", Item.class, false); - Iterable loaded = testTransaction.loadObjects(Item.class, Optional.of(filter), Optional.empty(), Optional.empty(), mockScope); + Iterable loaded = testTransaction.loadObjects(EntityProjection.builder() + .type(Item.class) + .filterExpression(filter) + .build(), mockScope); assertListContains(loaded, Lists.newArrayList()); - verify(wrappedTransaction, never()).loadObjects(any(), any(), any(), any(), any()); + verify(wrappedTransaction, never()).loadObjects(any(), any()); } @Test @@ -142,10 +153,13 @@ public void testPrefixPredicateWithInMemoryFiltering() throws Exception { //Case sensitive query against case insensitive index must lowercase FilterExpression filter = filterParser.parseFilterExpression("name==dru*", Item.class, false); - Iterable loaded = testTransaction.loadObjects(Item.class, Optional.of(filter), Optional.empty(), Optional.empty(), mockScope); + Iterable loaded = testTransaction.loadObjects(EntityProjection.builder() + .type(Item.class) + .filterExpression(filter) + .build(), mockScope); assertListContains(loaded, Lists.newArrayList()); - verify(wrappedTransaction, never()).loadObjects(any(), any(), any(), any(), any()); + verify(wrappedTransaction, never()).loadObjects(any(), any()); } @Test @@ -156,10 +170,13 @@ public void testPrefixPredicatePhrase() throws Exception { //Case sensitive query against case insensitive index must lowercase FilterExpression filter = filterParser.parseFilterExpression("name=='snare\\ dru*'", Item.class, false); - Iterable loaded = testTransaction.loadObjects(Item.class, Optional.of(filter), Optional.empty(), Optional.empty(), mockScope); + Iterable loaded = testTransaction.loadObjects(EntityProjection.builder() + .type(Item.class) + .filterExpression(filter) + .build(), mockScope); assertListContains(loaded, Lists.newArrayList(1L)); - verify(wrappedTransaction, never()).loadObjects(any(), any(), any(), any(), any()); + verify(wrappedTransaction, never()).loadObjects(any(), any()); } @Test @@ -170,10 +187,13 @@ public void testTabCharacter() throws Exception { //Case sensitive query against case insensitive index must lowercase FilterExpression filter = filterParser.parseFilterExpression("name=='*est\tTa*'", Item.class, false); - Iterable loaded = testTransaction.loadObjects(Item.class, Optional.of(filter), Optional.empty(), Optional.empty(), mockScope); + Iterable loaded = testTransaction.loadObjects(EntityProjection.builder() + .type(Item.class) + .filterExpression(filter) + .build(), mockScope); assertListContains(loaded, Lists.newArrayList(7L)); - verify(wrappedTransaction, never()).loadObjects(any(), any(), any(), any(), any()); + verify(wrappedTransaction, never()).loadObjects(any(), any()); } @Test @@ -184,10 +204,13 @@ public void testContainsPredicate() throws Exception { //Case insensitive query against case insensitive index FilterExpression filter = filterParser.parseFilterExpression("name==*DrU*", Item.class, false); - Iterable loaded = testTransaction.loadObjects(Item.class, Optional.of(filter), Optional.empty(), Optional.empty(), mockScope); + Iterable loaded = testTransaction.loadObjects(EntityProjection.builder() + .type(Item.class) + .filterExpression(filter) + .build(), mockScope); assertListContains(loaded, Lists.newArrayList(1L, 3L)); - verify(wrappedTransaction, never()).loadObjects(any(), any(), any(), any(), any()); + verify(wrappedTransaction, never()).loadObjects(any(), any()); } @Test @@ -197,10 +220,13 @@ public void testPredicateConjunction() throws Exception { FilterExpression filter = filterParser.parseFilterExpression("name==drum*;description==brass*", Item.class, false); - Iterable loaded = testTransaction.loadObjects(Item.class, Optional.of(filter), Optional.empty(), Optional.empty(), mockScope); + Iterable loaded = testTransaction.loadObjects(EntityProjection.builder() + .type(Item.class) + .filterExpression(filter) + .build(), mockScope); assertListContains(loaded, Lists.newArrayList(1L)); - verify(wrappedTransaction, never()).loadObjects(any(), any(), any(), any(), any()); + verify(wrappedTransaction, never()).loadObjects(any(), any()); } @Test @@ -210,12 +236,15 @@ public void testNonIndexedPredicate() throws Exception { FilterExpression filter = filterParser.parseFilterExpression("price==123;description==brass*", Item.class, false); - Iterable loaded = testTransaction.loadObjects(Item.class, Optional.of(filter), Optional.empty(), Optional.empty(), mockScope); + Iterable loaded = testTransaction.loadObjects(EntityProjection.builder() + .type(Item.class) + .filterExpression(filter) + .build(), mockScope); assertListContains(loaded, Lists.newArrayList()); /* This query should hit the underlying store */ - verify(wrappedTransaction, times(1)).loadObjects(any(), any(), any(), any(), any()); + verify(wrappedTransaction, times(1)).loadObjects(any(), any()); } @Test @@ -225,10 +254,13 @@ public void testPredicateDisjunction() throws Exception { FilterExpression filter = filterParser.parseFilterExpression("name==drum*,description==ride*", Item.class, false); - Iterable loaded = testTransaction.loadObjects(Item.class, Optional.of(filter), Optional.empty(), Optional.empty(), mockScope); + Iterable loaded = testTransaction.loadObjects(EntityProjection.builder() + .type(Item.class) + .filterExpression(filter) + .build(), mockScope); assertListContains(loaded, Lists.newArrayList(1L, 2L, 3L)); - verify(wrappedTransaction, never()).loadObjects(any(), any(), any(), any(), any()); + verify(wrappedTransaction, never()).loadObjects(any(), any()); } @Test @@ -243,10 +275,13 @@ public void testSortingAscending() throws Exception { FilterExpression filter = filterParser.parseFilterExpression("name==cymbal*", Item.class, false); - Iterable loaded = testTransaction.loadObjects(Item.class, Optional.of(filter), Optional.of(sorting), Optional.empty(), mockScope); + Iterable loaded = testTransaction.loadObjects(EntityProjection.builder() + .type(Item.class) + .filterExpression(filter) + .build(), mockScope); assertListContains(loaded, Lists.newArrayList(4L, 5L, 2L)); - verify(wrappedTransaction, never()).loadObjects(any(), any(), any(), any(), any()); + verify(wrappedTransaction, never()).loadObjects(any(), any()); } @Test @@ -261,10 +296,14 @@ public void testSortingDescending() throws Exception { FilterExpression filter = filterParser.parseFilterExpression("name==cymbal*", Item.class, false); - Iterable loaded = testTransaction.loadObjects(Item.class, Optional.of(filter), Optional.of(sorting), Optional.empty(), mockScope); + Iterable loaded = testTransaction.loadObjects(EntityProjection.builder() + .type(Item.class) + .filterExpression(filter) + .sorting(sorting) + .build(), mockScope); assertListMatches(loaded, Lists.newArrayList(2L, 5L, 4L)); - verify(wrappedTransaction, never()).loadObjects(any(), any(), any(), any(), any()); + verify(wrappedTransaction, never()).loadObjects(any(), any()); } @Test @@ -281,12 +320,16 @@ public void testPaginationPageOne() throws Exception { FilterExpression filter = filterParser.parseFilterExpression("name==cymbal*", Item.class, false); - Iterable loaded = testTransaction.loadObjects(Item.class, Optional.of(filter), Optional.of(sorting), Optional.of(pagination), mockScope); + Iterable loaded = testTransaction.loadObjects(EntityProjection.builder() + .type(Item.class) + .filterExpression(filter) + .sorting(sorting) + .pagination(pagination) + .build(), mockScope); assertListMatches(loaded, Lists.newArrayList(2L)); - assertEquals(3, pagination.getPageTotals()); - - verify(wrappedTransaction, never()).loadObjects(any(), any(), any(), any(), any()); + assertEquals(pagination.getPageTotals(), 3); + verify(wrappedTransaction, never()).loadObjects(any(), any()); } @Test @@ -303,11 +346,16 @@ public void testPaginationPageTwo() throws Exception { FilterExpression filter = filterParser.parseFilterExpression("name==cymbal*", Item.class, false); - Iterable loaded = testTransaction.loadObjects(Item.class, Optional.of(filter), Optional.of(sorting), Optional.of(pagination), mockScope); + Iterable loaded = testTransaction.loadObjects(EntityProjection.builder() + .type(Item.class) + .filterExpression(filter) + .sorting(sorting) + .pagination(pagination) + .build(), mockScope); assertListMatches(loaded, Lists.newArrayList(5L)); - assertEquals(3, pagination.getPageTotals()); - verify(wrappedTransaction, never()).loadObjects(any(), any(), any(), any(), any()); + assertEquals(pagination.getPageTotals(), 3); + verify(wrappedTransaction, never()).loadObjects(any(), any()); } @Test diff --git a/elide-datastore/pom.xml b/elide-datastore/pom.xml index e8197c8d27..0b908f44d5 100644 --- a/elide-datastore/pom.xml +++ b/elide-datastore/pom.xml @@ -14,7 +14,7 @@ com.yahoo.elide elide-parent-pom - 4.6.2-SNAPSHOT + 5.0.0-pr6-SNAPSHOT @@ -44,6 +44,7 @@ + elide-datastore-aggregation elide-datastore-hibernate elide-datastore-hibernate5 elide-datastore-hibernate3 @@ -59,7 +60,7 @@ com.yahoo.elide elide-core - 4.6.2-SNAPSHOT + 5.0.0-pr6-SNAPSHOT org.hibernate diff --git a/elide-example-models/pom.xml b/elide-example-models/pom.xml index 9c21e53b25..ed892206c9 100644 --- a/elide-example-models/pom.xml +++ b/elide-example-models/pom.xml @@ -14,7 +14,7 @@ elide-parent-pom com.yahoo.elide - 4.6.2-SNAPSHOT + 5.0.0-pr6-SNAPSHOT diff --git a/elide-example/elide-blog-example-resteasy/pom.xml b/elide-example/elide-blog-example-resteasy/pom.xml index 188fba2129..f4fd5fda9e 100644 --- a/elide-example/elide-blog-example-resteasy/pom.xml +++ b/elide-example/elide-blog-example-resteasy/pom.xml @@ -13,7 +13,7 @@ com.yahoo.elide elide-example-parent-pom - 4.6.2-SNAPSHOT + 5.0.0-pr6-SNAPSHOT @@ -52,12 +52,12 @@ com.yahoo.elide elide-core - 4.6.2-SNAPSHOT + 5.0.0-pr6-SNAPSHOT com.yahoo.elide elide-datastore-hibernate5 - 4.6.2-SNAPSHOT + 5.0.0-pr6-SNAPSHOT diff --git a/elide-example/elide-blog-example/pom.xml b/elide-example/elide-blog-example/pom.xml index 5dddaaad31..46ce1d361e 100644 --- a/elide-example/elide-blog-example/pom.xml +++ b/elide-example/elide-blog-example/pom.xml @@ -10,7 +10,7 @@ Elide Example: Hibernate5 API with Security Elide example using javax.persistence, MySQL and Elide Security com.yahoo.elide - 4.6.2-SNAPSHOT + 5.0.0-pr6-SNAPSHOT https://github.com/yahoo/elide @@ -59,17 +59,17 @@ com.yahoo.elide elide-annotations - 4.6.2-SNAPSHOT + 5.0.0-pr6-SNAPSHOT com.yahoo.elide elide-standalone - 4.6.2-SNAPSHOT + 5.0.0-pr6-SNAPSHOT com.yahoo.elide elide-test-helpers - 4.6.2-SNAPSHOT + 5.0.0-pr6-SNAPSHOT org.antlr diff --git a/elide-example/elide-hibernate3-mysql-example/pom.xml b/elide-example/elide-hibernate3-mysql-example/pom.xml index e891a830ef..37f782296b 100644 --- a/elide-example/elide-hibernate3-mysql-example/pom.xml +++ b/elide-example/elide-hibernate3-mysql-example/pom.xml @@ -13,7 +13,7 @@ com.yahoo.elide elide-example-parent-pom - 4.6.2-SNAPSHOT + 5.0.0-pr6-SNAPSHOT @@ -25,12 +25,12 @@ com.yahoo.elide elide-core - 4.6.2-SNAPSHOT + 5.0.0-pr6-SNAPSHOT com.yahoo.elide elide-datastore-hibernate3 - 4.6.2-SNAPSHOT + 5.0.0-pr6-SNAPSHOT diff --git a/elide-example/pom.xml b/elide-example/pom.xml index c19762b997..2596e3b147 100644 --- a/elide-example/pom.xml +++ b/elide-example/pom.xml @@ -14,7 +14,7 @@ com.yahoo.elide elide-parent-pom - 4.6.2-SNAPSHOT + 5.0.0-pr6-SNAPSHOT @@ -48,7 +48,7 @@ com.yahoo.elide elide-core - 4.6.2-SNAPSHOT + 5.0.0-pr6-SNAPSHOT org.apache.logging.log4j diff --git a/elide-graphql/pom.xml b/elide-graphql/pom.xml index a531b57cdc..6880a83fce 100644 --- a/elide-graphql/pom.xml +++ b/elide-graphql/pom.xml @@ -11,7 +11,7 @@ com.yahoo.elide elide-parent-pom - 4.6.2-SNAPSHOT + 5.0.0-pr6-SNAPSHOT @@ -44,12 +44,12 @@ com.yahoo.elide elide-core - 4.6.2-SNAPSHOT + 5.0.0-pr6-SNAPSHOT com.yahoo.elide elide-test-helpers - 4.6.2-SNAPSHOT + 5.0.0-pr6-SNAPSHOT com.fasterxml.jackson.core diff --git a/elide-graphql/src/main/java/com/yahoo/elide/graphql/Entity.java b/elide-graphql/src/main/java/com/yahoo/elide/graphql/Entity.java index e48dcbf01e..67e88cf9ea 100644 --- a/elide-graphql/src/main/java/com/yahoo/elide/graphql/Entity.java +++ b/elide-graphql/src/main/java/com/yahoo/elide/graphql/Entity.java @@ -9,6 +9,7 @@ import com.yahoo.elide.core.EntityDictionary; import com.yahoo.elide.core.PersistentResource; import com.yahoo.elide.core.RequestScope; +import com.yahoo.elide.request.EntityProjection; import lombok.AllArgsConstructor; import lombok.Getter; @@ -37,11 +38,14 @@ public class Entity { * Class constructor. * @param parentResource parent entity * @param data entity data - * @param entityClass binding entity class + * @param entityClass entity class * @param requestScope the request context object */ - public Entity(Optional parentResource, Map data, - Class entityClass, RequestScope requestScope) { + public Entity( + Optional parentResource, + Map data, + Class entityClass, + RequestScope requestScope) { this.parentResource = parentResource; this.data = data; this.entityClass = entityClass; @@ -90,24 +94,33 @@ private void setRelationships() { this.relationships = new LinkedHashSet<>(); EntityDictionary dictionary = this.requestScope.getDictionary(); - for (Map.Entry entry : this.data.entrySet()) { - if (dictionary.isRelation(this.entityClass, entry.getKey())) { - Set entitySet = new LinkedHashSet<>(); - Class loadClass = dictionary.getParameterizedType(this.entityClass, entry.getKey()); - Boolean isToOne = dictionary.getRelationshipType(this.entityClass, entry.getKey()).isToOne(); - if (isToOne) { - entitySet.add(new Entity(Optional.of(this), - ((Map) entry.getValue()), - loadClass, - this.requestScope)); - } else { - for (Map row : (List>) entry.getValue()) { - entitySet.add(new Entity(Optional.of(this), row, loadClass, this.requestScope)); + this.data.entrySet().stream() + .filter(entry -> dictionary.isRelation(this.entityClass, entry.getKey())) + .forEach(entry -> { + String relationshipName = entry.getKey(); + Class relationshipClass = + dictionary.getParameterizedType(this.entityClass, relationshipName); + + Set relationshipEntities = new LinkedHashSet<>(); + + // if the relationship is ToOne, entry.getValue() should be a single map + if (dictionary.getRelationshipType(this.entityClass, relationshipName).isToOne()) { + relationshipEntities.add(new Entity( + Optional.of(this), + ((Map) entry.getValue()), + relationshipClass, + this.requestScope)); + } else { + for (Map row : (List>) entry.getValue()) { + relationshipEntities.add(new Entity( + Optional.of(this), + row, + relationshipClass, + this.requestScope)); + } } - } - this.relationships.add(new Relationship(entry.getKey(), entitySet)); - } - } + this.relationships.add(new Relationship(relationshipName, relationshipEntities)); + }); } } @@ -173,8 +186,16 @@ public void setId() { * @return {@link PersistentResource} object */ public PersistentResource toPersistentResource() { - return this.data == null ? null : PersistentResource.loadRecord(this.entityClass, - getId().orElse(null), - this.requestScope); + return this.data == null + ? null + : PersistentResource.loadRecord(getProjection(), getId().orElse(null), this.requestScope); + } + + /** + * Get a projection for this entity class. Used for querying inserted entities. + * @return {@link EntityProjection} object + */ + public EntityProjection getProjection() { + return EntityProjection.builder().type(entityClass).build(); } } diff --git a/elide-graphql/src/main/java/com/yahoo/elide/graphql/Environment.java b/elide-graphql/src/main/java/com/yahoo/elide/graphql/Environment.java index cacb21a007..8693fc4446 100644 --- a/elide-graphql/src/main/java/com/yahoo/elide/graphql/Environment.java +++ b/elide-graphql/src/main/java/com/yahoo/elide/graphql/Environment.java @@ -42,7 +42,7 @@ public class Environment { public Environment(DataFetchingEnvironment environment) { Map args = environment.getArguments(); - requestScope = (GraphQLRequestScope) environment.getContext(); + requestScope = environment.getContext(); filters = Optional.ofNullable((String) args.get(ModelBuilder.ARGUMENT_FILTER)); offset = Optional.ofNullable((String) args.get(ModelBuilder.ARGUMENT_AFTER)); diff --git a/elide-graphql/src/main/java/com/yahoo/elide/graphql/GraphQLConversionUtils.java b/elide-graphql/src/main/java/com/yahoo/elide/graphql/GraphQLConversionUtils.java index a00fd0b338..8d5773744c 100644 --- a/elide-graphql/src/main/java/com/yahoo/elide/graphql/GraphQLConversionUtils.java +++ b/elide-graphql/src/main/java/com/yahoo/elide/graphql/GraphQLConversionUtils.java @@ -6,6 +6,7 @@ package com.yahoo.elide.graphql; +import static graphql.schema.GraphQLArgument.newArgument; import static graphql.schema.GraphQLEnumType.newEnum; import static graphql.schema.GraphQLFieldDefinition.newFieldDefinition; import static graphql.schema.GraphQLInputObjectField.newInputObjectField; @@ -19,6 +20,7 @@ import graphql.Scalars; import graphql.schema.DataFetcher; +import graphql.schema.GraphQLArgument; import graphql.schema.GraphQLEnumType; import graphql.schema.GraphQLFieldDefinition; import graphql.schema.GraphQLInputObjectField; @@ -34,7 +36,9 @@ import java.util.Collection; import java.util.Date; import java.util.HashMap; +import java.util.List; import java.util.Map; +import java.util.stream.Collectors; /** * Contains methods that convert from a class to a GraphQL input or query type. @@ -443,6 +447,42 @@ public GraphQLInputObjectType classToInputObject(Class clazz) { return object; } + /** + * Build an Argument list object for the given attribute. + * @param entityClass The Entity class to which this attribute belongs to. + * @param attribute The name of the attribute. + * @param fetcher The data fetcher to associated with the newly created GraphQL Query Type. + * @return Newly created GraphQLArgument Collection for the given attribute. + */ + public List attributeArgumentToQueryObject(Class entityClass, + String attribute, + DataFetcher fetcher) { + return attributeArgumentToQueryObject(entityClass, attribute, fetcher, entityDictionary); + } + + /** + * Build an Argument list object for the given attribute + * @param entityClass The Entity class to which this attribute belongs to. + * @param attribute The name of the attribute. + * @param fetcher The data fetcher to associated with the newly created GraphQL Query Type + * @param dictionary The dictionary that contains the runtime type information for the parent class. + * @return Newly created GraphQLArgument Collection for the given attribute + */ + public List attributeArgumentToQueryObject(Class entityClass, + String attribute, + DataFetcher fetcher, + EntityDictionary dictionary) { + return dictionary.getAttributeArguments(entityClass, attribute) + .stream() + .map(argumentType -> newArgument() + .name(argumentType.getName()) + .type(fetchScalarOrObjectInput(argumentType.getType())) + .build()) + .collect(Collectors.toList()); + + } + + private GraphQLOutputType fetchScalarOrObjectOutput(Class conversionClass, DataFetcher fetcher) { /* If class is enum, provide enum type */ diff --git a/elide-graphql/src/main/java/com/yahoo/elide/graphql/GraphQLEndpoint.java b/elide-graphql/src/main/java/com/yahoo/elide/graphql/GraphQLEndpoint.java index 35a9b87ba1..c6048d487a 100644 --- a/elide-graphql/src/main/java/com/yahoo/elide/graphql/GraphQLEndpoint.java +++ b/elide-graphql/src/main/java/com/yahoo/elide/graphql/GraphQLEndpoint.java @@ -8,7 +8,6 @@ import com.yahoo.elide.Elide; import com.yahoo.elide.ElideResponse; import com.yahoo.elide.resources.DefaultOpaqueUserFunction; - import lombok.extern.slf4j.Slf4j; import java.util.function.Function; @@ -60,7 +59,6 @@ public GraphQLEndpoint( public Response post( @Context SecurityContext securityContext, String graphQLDocument) { - ElideResponse response = runner.run(graphQLDocument, getUser.apply(securityContext)); return Response.status(response.getResponseCode()).entity(response.getBody()).build(); } diff --git a/elide-graphql/src/main/java/com/yahoo/elide/graphql/GraphQLRequestScope.java b/elide-graphql/src/main/java/com/yahoo/elide/graphql/GraphQLRequestScope.java index 6f31328804..fcaa83d3b6 100644 --- a/elide-graphql/src/main/java/com/yahoo/elide/graphql/GraphQLRequestScope.java +++ b/elide-graphql/src/main/java/com/yahoo/elide/graphql/GraphQLRequestScope.java @@ -8,13 +8,12 @@ import com.yahoo.elide.ElideSettings; import com.yahoo.elide.core.DataStoreTransaction; import com.yahoo.elide.core.RequestScope; +import com.yahoo.elide.graphql.parser.GraphQLProjectionInfo; import com.yahoo.elide.security.User; import lombok.Getter; - import java.util.HashMap; import java.util.Map; - import javax.ws.rs.core.MultivaluedHashMap; /** @@ -23,13 +22,23 @@ public class GraphQLRequestScope extends RequestScope { @Getter private final Map totalRecordCounts = new HashMap<>(); - public GraphQLRequestScope(DataStoreTransaction transaction, - User user, - ElideSettings elideSettings) { + @Getter + private final GraphQLProjectionInfo projectionInfo; + + public GraphQLRequestScope( + DataStoreTransaction transaction, + User user, + ElideSettings elideSettings, + GraphQLProjectionInfo projectionInfo + ) { // TODO: We're going to break out the two request scopes. `RequestScope` should become an interface and // we should have a GraphQLRequestScope and a JSONAPIRequestScope. // TODO: What should mutate multiple entity value be? There is a problem with this setting in practice. // Namely, we don't filter or paginate in the data store. super("/", null, transaction, user, new MultivaluedHashMap<>(), elideSettings); + this.projectionInfo = projectionInfo; + + // Entity Projection is retrieved from projectionInfo. + this.setEntityProjection(null); } } diff --git a/elide-graphql/src/main/java/com/yahoo/elide/graphql/KeyWord.java b/elide-graphql/src/main/java/com/yahoo/elide/graphql/KeyWord.java new file mode 100644 index 0000000000..4e1b69f130 --- /dev/null +++ b/elide-graphql/src/main/java/com/yahoo/elide/graphql/KeyWord.java @@ -0,0 +1,47 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ + +package com.yahoo.elide.graphql; + +import lombok.Getter; + +/** + * Key words used in graphql parsing. + */ +public enum KeyWord { + NODE("node"), + EDGES("edges"), + PAGE_INFO("pageInfo"), + PAGE_INFO_HAS_NEXT_PAGE("hasNextPage"), + PAGE_INFO_START_CURSOR("startCursor"), + PAGE_INFO_END_CURSOR("endCursor"), + PAGE_INFO_TOTAL_RECORDS("totalRecords"), + TYPENAME("__typename"), + SCHEMA("__schema"), + TYPE("__type"), + UNKNOWN("unknown"); + + @Getter + private String name; + + KeyWord(String name) { + this.name = name; + } + + public boolean equals(String name) { + return this.name.equals(name); + } + + public static KeyWord byName(String value) { + for (KeyWord keyWord : KeyWord.values()) { + if (keyWord.equals(value)) { + return keyWord; + } + } + + return UNKNOWN; + } +} diff --git a/elide-graphql/src/main/java/com/yahoo/elide/graphql/ModelBuilder.java b/elide-graphql/src/main/java/com/yahoo/elide/graphql/ModelBuilder.java index 06947b5409..640aa98c6a 100644 --- a/elide-graphql/src/main/java/com/yahoo/elide/graphql/ModelBuilder.java +++ b/elide-graphql/src/main/java/com/yahoo/elide/graphql/ModelBuilder.java @@ -242,7 +242,6 @@ private GraphQLObjectType buildQueryObject(Class entityClass) { .name("_node__" + entityName); String id = dictionary.getIdFieldName(entityClass); - /* our id types are DeferredId objects (not Scalars.GraphQLID) */ builder.field(newFieldDefinition() .name(id) @@ -255,9 +254,10 @@ private GraphQLObjectType buildQueryObject(Class entityClass) { continue; } - log.debug("Building query attribute {} {} for entity {}", + log.debug("Building query attribute {} {} with arguments {} for entity {}", attribute, attributeClass.getName(), + dictionary.getAttributeArguments(attributeClass, attribute).toString(), entityClass.getName()); GraphQLType attributeType = @@ -269,6 +269,7 @@ private GraphQLObjectType buildQueryObject(Class entityClass) { builder.field(newFieldDefinition() .name(attribute) + .argument(generator.attributeArgumentToQueryObject(entityClass, attribute, dataFetcher)) .dataFetcher(dataFetcher) .type((GraphQLOutputType) attributeType) ); @@ -357,9 +358,11 @@ private GraphQLInputType buildInputObjectStub(Class clazz) { builder.name(entityName + ARGUMENT_INPUT); String id = dictionary.getIdFieldName(clazz); - builder.field(newInputObjectField() - .name(id) - .type(Scalars.GraphQLID)); + if (id != null) { + builder.field(newInputObjectField() + .name(id) + .type(Scalars.GraphQLID)); + } for (String attribute : dictionary.getAttributes(clazz)) { Class attributeClass = dictionary.getType(clazz, attribute); diff --git a/elide-graphql/src/main/java/com/yahoo/elide/graphql/PersistentResourceFetcher.java b/elide-graphql/src/main/java/com/yahoo/elide/graphql/PersistentResourceFetcher.java index 627a88233a..44713a8257 100644 --- a/elide-graphql/src/main/java/com/yahoo/elide/graphql/PersistentResourceFetcher.java +++ b/elide-graphql/src/main/java/com/yahoo/elide/graphql/PersistentResourceFetcher.java @@ -8,18 +8,14 @@ import static com.yahoo.elide.graphql.ModelBuilder.ARGUMENT_OPERATION; -import com.yahoo.elide.ElideSettings; import com.yahoo.elide.core.EntityDictionary; import com.yahoo.elide.core.PersistentResource; import com.yahoo.elide.core.RequestScope; import com.yahoo.elide.core.exceptions.InvalidObjectIdentifierException; -import com.yahoo.elide.core.exceptions.InvalidPredicateException; import com.yahoo.elide.core.exceptions.InvalidValueException; -import com.yahoo.elide.core.filter.dialect.ParseException; -import com.yahoo.elide.core.filter.expression.FilterExpression; -import com.yahoo.elide.core.pagination.Pagination; -import com.yahoo.elide.core.sort.Sorting; import com.yahoo.elide.graphql.containers.ConnectionContainer; +import com.yahoo.elide.request.EntityProjection; +import com.yahoo.elide.request.Relationship; import com.google.common.collect.Sets; @@ -30,10 +26,8 @@ import graphql.schema.DataFetchingEnvironment; import graphql.schema.GraphQLType; import lombok.extern.slf4j.Slf4j; - import java.util.ArrayDeque; import java.util.ArrayList; -import java.util.Arrays; import java.util.Collections; import java.util.LinkedHashSet; import java.util.List; @@ -43,21 +37,15 @@ import java.util.Queue; import java.util.Set; import java.util.stream.Collectors; - +import javax.validation.constraints.NotNull; import javax.ws.rs.BadRequestException; -import javax.ws.rs.core.MultivaluedHashMap; -import javax.ws.rs.core.MultivaluedMap; /** * Invoked by GraphQL Java to fetch/mutate data from Elide. */ @Slf4j public class PersistentResourceFetcher implements DataFetcher { - private final ElideSettings settings; - - public PersistentResourceFetcher(ElideSettings settings) { - this.settings = settings; - } + public PersistentResourceFetcher() { } /** * Override graphql-java's {@link DataFetcher} get method to execute @@ -157,27 +145,18 @@ private Object fetchObjects(Environment context) { /** * Fetches a root-level entity. - * @param context Context for request * @param requestScope Request scope - * @param entityClass Entity class + * @param projection constructed entityProjection for a class * @param ids List of ids (can be NULL) - * @param sort Sort by ASC/DESC - * @param offset Pagination offset argument - * @param first Pagination first argument - * @param filters Filter params - * @param generateTotals True if page totals should be generated for this type, false otherwise * @return {@link PersistentResource} object(s) */ - public ConnectionContainer fetchObject(Environment context, RequestScope requestScope, Class entityClass, - Optional> ids, Optional sort, - Optional offset, Optional first, - Optional filters, boolean generateTotals) { + public ConnectionContainer fetchObject( + RequestScope requestScope, + EntityProjection projection, + Optional> ids + ) { EntityDictionary dictionary = requestScope.getDictionary(); - String typeName = dictionary.getJsonAliasFor(entityClass); - - Optional pagination = buildPagination(first, offset, generateTotals); - Optional sorting = buildSorting(sort); - Optional filter = buildFilter(typeName, filters, requestScope); + String typeName = dictionary.getJsonAliasFor(projection.getType()); /* fetching a collection */ Set records = ids.map((idList) -> { @@ -186,67 +165,53 @@ public ConnectionContainer fetchObject(Environment context, RequestScope request throw new BadRequestException("Empty list passed to ids"); } - return PersistentResource.loadRecords(entityClass, idList, - filter, sorting, pagination, requestScope); - }).orElseGet(() -> PersistentResource.loadRecords( - entityClass, /* Empty list of IDs */ new ArrayList<>(), filter, sorting, pagination, requestScope - )); + return PersistentResource.loadRecords(projection, idList, requestScope); + }).orElseGet(() -> PersistentResource.loadRecords(projection, new ArrayList<>(), requestScope)); - return new ConnectionContainer(records, pagination, typeName); + return new ConnectionContainer(records, Optional.ofNullable(projection.getPagination()), typeName); } /** * Fetches a relationship for a top-level entity. * - * @param context Request context * @param parentResource Parent object - * @param fieldName Field type + * @param relationship constructed relationship object with entityProjection * @param ids List of ids - * @param offset Pagination offset - * @param first Pagination first - * @param filters Filter string - * @param generateTotals True if page totals should be generated for this type, false otherwise * @return persistence resource object(s) */ - public Object fetchRelationship(Environment context, - PersistentResource parentResource, - String fieldName, - Optional> ids, - Optional offset, - Optional first, - Optional sort, - Optional filters, - boolean generateTotals) { + public Object fetchRelationship( + PersistentResource parentResource, + @NotNull Relationship relationship, + Optional> ids + ) { EntityDictionary dictionary = parentResource.getRequestScope().getDictionary(); - Class entityClass = dictionary.getParameterizedType(parentResource.getObject(), fieldName); - String typeName = dictionary.getJsonAliasFor(entityClass); - - Optional pagination = buildPagination(first, offset, generateTotals); - Optional sorting = buildSorting(sort); - Optional filter = buildFilter(typeName, filters, parentResource.getRequestScope()); + Class relationshipClass = dictionary.getParameterizedType(parentResource.getObject(), relationship.getName()); + String relationshipType = dictionary.getJsonAliasFor(relationshipClass); - Set relations; + Set relationResources; if (ids.isPresent()) { - relations = parentResource.getRelation(fieldName, ids.get(), filter, sorting, pagination); + relationResources = parentResource.getRelation(ids.get(), relationship); } else { - relations = parentResource.getRelationCheckedFiltered(fieldName, - filter, sorting, pagination); + relationResources = parentResource.getRelationCheckedFiltered(relationship); } - return new ConnectionContainer(relations, pagination, typeName); + return new ConnectionContainer( + relationResources, + Optional.ofNullable(relationship.getProjection().getPagination()), + relationshipType); } private ConnectionContainer upsertObjects(Environment context) { return upsertOrUpdateObjects( context, - (entityObject) -> upsertObject(context, entityObject), + (entityObject) -> upsertObject(entityObject), RelationshipOp.UPSERT); } private ConnectionContainer updateObjects(Environment context) { return upsertOrUpdateObjects( context, - (entityObject) -> updateObject(context, entityObject), + (entityObject) -> updateObject(entityObject), RelationshipOp.UPDATE); } @@ -256,9 +221,12 @@ private ConnectionContainer updateObjects(Environment context) { * @param updateFunc controls the behavior of how the update (or upsert) is performed. * @return Connection object. */ - private ConnectionContainer upsertOrUpdateObjects(Environment context, - Executor updateFunc, - RelationshipOp operation) { + + private ConnectionContainer upsertOrUpdateObjects( + Environment context, + Executor updateFunc, + RelationshipOp operation + ) { /* sanity check for id and data argument w UPSERT/UPDATE */ if (context.ids.isPresent()) { throw new BadRequestException(operation + " must not include ids"); @@ -273,14 +241,18 @@ private ConnectionContainer upsertOrUpdateObjects(Environment context, if (context.isRoot()) { entityClass = dictionary.getEntityClass(context.field.getName()); } else { - entityClass = dictionary.getParameterizedType(context.parentResource.getResourceClass(), + assert context.parentResource != null; + entityClass = dictionary.getParameterizedType( + context.parentResource.getResourceClass(), context.field.getName()); } /* form entities */ Optional parentEntity; if (!context.isRoot()) { - parentEntity = Optional.of(new Entity(Optional.empty(), + assert context.parentResource != null; + parentEntity = Optional.of(new Entity( + Optional.empty(), null, context.parentResource.getResourceClass(), context.requestScope)); @@ -303,6 +275,7 @@ private ConnectionContainer upsertOrUpdateObjects(Environment context, PersistentResource childResource = entity.toPersistentResource(); if (!context.isRoot()) { /* add relation between parent and nested entity */ + assert context.parentResource != null; context.parentResource.addRelation(context.field.getName(), childResource); } } @@ -373,53 +346,45 @@ private PersistentResource updateRelationship(Entity entity) { /** * updates or creates existing/new entities - * @param context request context * @param entity Resource entity * @return {@link PersistentResource} object */ - private PersistentResource upsertObject(Environment context, Entity entity) { + private PersistentResource upsertObject(Entity entity) { Set attributes = entity.getAttributes(); Optional id = entity.getId(); RequestScope requestScope = entity.getRequestScope(); - PersistentResource upsertedResource; - PersistentResource parentResource; - if (!entity.getParentResource().isPresent()) { - parentResource = null; - } else { - parentResource = entity.getParentResource().get().toPersistentResource(); - } + PersistentResource upsertedResource; + + PersistentResource parentResource = !entity.getParentResource().isPresent() + ? null + : entity.getParentResource().get().toPersistentResource(); if (!id.isPresent()) { entity.setId(); id = entity.getId(); - upsertedResource = PersistentResource.createObject(parentResource, - entity.getEntityClass(), - requestScope, - id); + + upsertedResource = PersistentResource.createObject( + parentResource, entity.getEntityClass(), requestScope, id); } else { try { - Set loadedResource = fetchObject(context, requestScope, entity.getEntityClass(), - Optional.of(Arrays.asList(id.get())), - Optional.empty(), - Optional.empty(), - Optional.empty(), - Optional.empty(), - false).getPersistentResources(); - upsertedResource = IterableUtils.first(loadedResource); - - //The ID doesn't exist yet. Let's create the object. - } catch (InvalidObjectIdentifierException | InvalidValueException e) { - upsertedResource = PersistentResource.createObject(parentResource, - entity.getEntityClass(), + Set loadedResource = fetchObject( requestScope, - id); + entity.getProjection(), + Optional.of(Collections.singletonList(id.get())) + ).getPersistentResources(); + upsertedResource = loadedResource.iterator().next(); + + // The ID doesn't exist yet. Let's create the object. + } catch (InvalidObjectIdentifierException | InvalidValueException e) { + upsertedResource = PersistentResource.createObject( + parentResource, entity.getEntityClass(), requestScope, id); } } return updateAttributes(upsertedResource, entity, attributes); } - private PersistentResource updateObject(Environment context, Entity entity) { + private PersistentResource updateObject(Entity entity) { Set attributes = entity.getAttributes(); Optional id = entity.getId(); RequestScope requestScope = entity.getRequestScope(); @@ -427,15 +392,14 @@ private PersistentResource updateObject(Environment context, Entity entity) { if (!id.isPresent()) { throw new BadRequestException("UPDATE data objects must include ids"); + } else { + Set loadedResource = fetchObject( + requestScope, + entity.getProjection(), + Optional.of(Collections.singletonList(id.get())) + ).getPersistentResources(); + updatedResource = loadedResource.iterator().next(); } - Set loadedResource = fetchObject(context, requestScope, entity.getEntityClass(), - Optional.of(Arrays.asList(id.get())), - Optional.empty(), - Optional.empty(), - Optional.empty(), - Optional.empty(), - false).getPersistentResources(); - updatedResource = IterableUtils.first(loadedResource); return updateAttributes(updatedResource, entity, attributes); } @@ -552,34 +516,4 @@ private ConnectionContainer replaceObjects(Environment context) { } return upsertedObjects; } - - private Optional buildPagination(Optional first, - Optional offset, - boolean generateTotals) { - return Pagination.fromOffsetAndFirst(first, offset, generateTotals, settings); - } - - private Optional buildSorting(Optional sort) { - return sort.map(Sorting::parseSortRule); - } - - private Optional buildFilter(String typeName, - Optional filter, - RequestScope requestScope) { - // TODO: Refactor FilterDialect interfaces to accept string or List instead of (or in addition to?) - // query params. - return filter.map(filterStr -> { - MultivaluedMap queryParams = new MultivaluedHashMap() { - { - put("filter[" + typeName + "]", Arrays.asList(filterStr)); - } - }; - try { - return requestScope.getFilterDialect().parseTypedExpression(typeName, queryParams).get(typeName); - } catch (ParseException e) { - log.debug("Filter parse exception caught", e); - throw new InvalidPredicateException("Could not parse filter for type: " + typeName); - } - }); - } } diff --git a/elide-graphql/src/main/java/com/yahoo/elide/graphql/QueryRunner.java b/elide-graphql/src/main/java/com/yahoo/elide/graphql/QueryRunner.java index 6c783159be..ca51898fb1 100644 --- a/elide-graphql/src/main/java/com/yahoo/elide/graphql/QueryRunner.java +++ b/elide-graphql/src/main/java/com/yahoo/elide/graphql/QueryRunner.java @@ -15,6 +15,8 @@ import com.yahoo.elide.core.exceptions.HttpStatusException; import com.yahoo.elide.core.exceptions.InvalidEntityBodyException; import com.yahoo.elide.core.exceptions.TransactionException; +import com.yahoo.elide.graphql.parser.GraphQLEntityProjectionMaker; +import com.yahoo.elide.graphql.parser.GraphQLProjectionInfo; import com.yahoo.elide.security.User; import com.fasterxml.jackson.core.JsonProcessingException; @@ -64,7 +66,7 @@ public class QueryRunner { public QueryRunner(Elide elide) { this.elide = elide; - PersistentResourceFetcher fetcher = new PersistentResourceFetcher(elide.getElideSettings()); + PersistentResourceFetcher fetcher = new PersistentResourceFetcher(); ModelBuilder builder = new ModelBuilder(elide.getElideSettings().getDictionary(), fetcher); this.api = new GraphQL(builder.build()); @@ -142,8 +144,6 @@ private ElideResponse executeGraphQLRequest(ObjectMapper mapper, Object principa boolean isVerbose = false; try (DataStoreTransaction tx = elide.getDataStore().beginTransaction()) { final User user = tx.accessUser(principal); - GraphQLRequestScope requestScope = new GraphQLRequestScope(tx, user, elide.getElideSettings()); - isVerbose = requestScope.getPermissionExecutor().isVerbose(); if (!jsonDocument.has(QUERY)) { return ElideResponse.builder() @@ -151,9 +151,21 @@ private ElideResponse executeGraphQLRequest(ObjectMapper mapper, Object principa .body("A `query` key is required.") .build(); } - String query = jsonDocument.get(QUERY).asText(); + // get variables from request for constructing entityProjections + Map variables = new HashMap<>(); + if (jsonDocument.has(VARIABLES) && !jsonDocument.get(VARIABLES).isNull()) { + variables = mapper.convertValue(jsonDocument.get(VARIABLES), Map.class); + } + + GraphQLProjectionInfo projectionInfo = + new GraphQLEntityProjectionMaker(elide.getElideSettings(), variables).make(query); + GraphQLRequestScope requestScope = + new GraphQLRequestScope(tx, user, elide.getElideSettings(), projectionInfo); + + isVerbose = requestScope.getPermissionExecutor().isVerbose(); + // Logging all queries. It is recommended to put any private information that shouldn't be logged into // the "variables" section of your query. Variable values are not logged. log.info("Processing GraphQL query:\n{}", query); @@ -166,10 +178,7 @@ private ElideResponse executeGraphQLRequest(ObjectMapper mapper, Object principa executionInput.operationName(jsonDocument.get(OPERATION_NAME).asText()); } - if (jsonDocument.has(VARIABLES) && !jsonDocument.get(VARIABLES).isNull()) { - Map variables = mapper.convertValue(jsonDocument.get(VARIABLES), Map.class); - executionInput.variables(variables); - } + executionInput.variables(variables); ExecutionResult result = api.execute(executionInput); diff --git a/elide-graphql/src/main/java/com/yahoo/elide/graphql/containers/ConnectionContainer.java b/elide-graphql/src/main/java/com/yahoo/elide/graphql/containers/ConnectionContainer.java index 5524f21c4a..05885b4ef4 100644 --- a/elide-graphql/src/main/java/com/yahoo/elide/graphql/containers/ConnectionContainer.java +++ b/elide-graphql/src/main/java/com/yahoo/elide/graphql/containers/ConnectionContainer.java @@ -8,6 +8,7 @@ import com.yahoo.elide.core.PersistentResource; import com.yahoo.elide.core.pagination.Pagination; import com.yahoo.elide.graphql.Environment; +import com.yahoo.elide.graphql.KeyWord; import com.yahoo.elide.graphql.PersistentResourceFetcher; import lombok.AllArgsConstructor; @@ -29,19 +30,16 @@ public class ConnectionContainer implements GraphQLContainer { // Refers to the type of persistentResources @Getter private final String typeName; - public static final String EDGES_KEYWORD = "edges"; - public static final String PAGE_INFO_KEYWORD = "pageInfo"; - @Override public Object processFetch(Environment context, PersistentResourceFetcher fetcher) { String fieldName = context.field.getName(); - switch (fieldName) { - case EDGES_KEYWORD: + switch (KeyWord.byName(fieldName)) { + case EDGES: return getPersistentResources().stream() .map(EdgesContainer::new) .collect(Collectors.toList()); - case PAGE_INFO_KEYWORD: + case PAGE_INFO: return new PageInfoContainer(this); default: break; diff --git a/elide-graphql/src/main/java/com/yahoo/elide/graphql/containers/EdgesContainer.java b/elide-graphql/src/main/java/com/yahoo/elide/graphql/containers/EdgesContainer.java index 64a1a0d270..1b11c8df4b 100644 --- a/elide-graphql/src/main/java/com/yahoo/elide/graphql/containers/EdgesContainer.java +++ b/elide-graphql/src/main/java/com/yahoo/elide/graphql/containers/EdgesContainer.java @@ -5,6 +5,8 @@ */ package com.yahoo.elide.graphql.containers; +import static com.yahoo.elide.graphql.KeyWord.NODE; + import com.yahoo.elide.core.PersistentResource; import com.yahoo.elide.graphql.Environment; import com.yahoo.elide.graphql.PersistentResourceFetcher; @@ -21,14 +23,12 @@ public class EdgesContainer implements PersistentResourceContainer, GraphQLContainer { @Getter private final PersistentResource persistentResource; - private static final String NODE_KEYWORD = "node"; - @Override public Object processFetch(Environment context, PersistentResourceFetcher fetcher) { String fieldName = context.field.getName(); // TODO: Cursor - if (NODE_KEYWORD.equals(fieldName)) { + if (NODE.equals(fieldName)) { return new NodeContainer(context.parentResource); } diff --git a/elide-graphql/src/main/java/com/yahoo/elide/graphql/containers/NodeContainer.java b/elide-graphql/src/main/java/com/yahoo/elide/graphql/containers/NodeContainer.java index 5e66afc576..f246851e54 100644 --- a/elide-graphql/src/main/java/com/yahoo/elide/graphql/containers/NodeContainer.java +++ b/elide-graphql/src/main/java/com/yahoo/elide/graphql/containers/NodeContainer.java @@ -5,13 +5,13 @@ */ package com.yahoo.elide.graphql.containers; -import static com.yahoo.elide.graphql.containers.RootContainer.requestContainsPageInfo; - import com.yahoo.elide.core.EntityDictionary; import com.yahoo.elide.core.PersistentResource; import com.yahoo.elide.graphql.DeferredId; import com.yahoo.elide.graphql.Environment; import com.yahoo.elide.graphql.PersistentResourceFetcher; +import com.yahoo.elide.request.Attribute; +import com.yahoo.elide.request.Relationship; import lombok.AllArgsConstructor; import lombok.Getter; @@ -37,7 +37,9 @@ public Object processFetch(Environment context, PersistentResourceFetcher fetche String idFieldName = dictionary.getIdFieldName(parentClass); if (dictionary.isAttribute(parentClass, fieldName)) { /* fetch attribute properties */ - Object attribute = context.parentResource.getAttribute(fieldName); + Attribute requested = context.requestScope.getProjectionInfo() + .getAttributeMap().getOrDefault(context.field.getSourceLocation(), null); + Object attribute = context.parentResource.getAttribute(requested); if (attribute instanceof Map) { return ((Map) attribute).entrySet().stream() .map(MapEntryContainer::new) @@ -46,10 +48,18 @@ public Object processFetch(Environment context, PersistentResourceFetcher fetche return attribute; } if (dictionary.isRelation(parentClass, fieldName)) { /* fetch relationship properties */ - boolean generateTotals = requestContainsPageInfo(context.field); - return fetcher.fetchRelationship(context, context.parentResource, - fieldName, context.ids, context.offset, context.first, context.sort, context.filters, - generateTotals); + // get the relationship from constructed projections + Relationship relationship = context.requestScope + .getProjectionInfo() + .getRelationshipMap() + .getOrDefault(context.field.getSourceLocation(), null); + + if (relationship == null) { + throw new BadRequestException( + "Relationship doesn't have projection " + context.parentResource.getType() + "." + fieldName); + } + + return fetcher.fetchRelationship(context.parentResource, relationship, context.ids); } if (Objects.equals(idFieldName, fieldName)) { return new DeferredId(context.parentResource); diff --git a/elide-graphql/src/main/java/com/yahoo/elide/graphql/containers/PageInfoContainer.java b/elide-graphql/src/main/java/com/yahoo/elide/graphql/containers/PageInfoContainer.java index bbbfac9074..4e0bd8b7c3 100644 --- a/elide-graphql/src/main/java/com/yahoo/elide/graphql/containers/PageInfoContainer.java +++ b/elide-graphql/src/main/java/com/yahoo/elide/graphql/containers/PageInfoContainer.java @@ -8,14 +8,13 @@ import com.yahoo.elide.core.PersistentResource; import com.yahoo.elide.core.pagination.Pagination; import com.yahoo.elide.graphql.Environment; +import com.yahoo.elide.graphql.KeyWord; import com.yahoo.elide.graphql.PersistentResourceFetcher; import lombok.Getter; - import java.util.List; import java.util.Optional; import java.util.stream.Collectors; - import javax.ws.rs.BadRequestException; /** @@ -24,12 +23,6 @@ public class PageInfoContainer implements GraphQLContainer { @Getter private final ConnectionContainer connectionContainer; - // Page info keywords - private static final String PAGE_INFO_HAS_NEXT_PAGE_KEYWORD = "hasNextPage"; - private static final String PAGE_INFO_START_CURSOR_KEYWORD = "startCursor"; - private static final String PAGE_INFO_END_CURSOR_KEYWORD = "endCursor"; - private static final String PAGE_INFO_TOTAL_RECORDS_KEYWORD = "totalRecords"; - public PageInfoContainer(ConnectionContainer connectionContainer) { this.connectionContainer = connectionContainer; } @@ -46,17 +39,17 @@ public Object processFetch(Environment context, PersistentResourceFetcher fetche .collect(Collectors.toList()); return pagination.map(pageValue -> { - switch (fieldName) { - case PAGE_INFO_HAS_NEXT_PAGE_KEYWORD: { + switch (KeyWord.byName(fieldName)) { + case PAGE_INFO_HAS_NEXT_PAGE: { int numResults = ids.size(); int nextOffset = numResults + pageValue.getOffset(); return nextOffset < pageValue.getPageTotals(); } - case PAGE_INFO_START_CURSOR_KEYWORD: + case PAGE_INFO_START_CURSOR: return pageValue.getOffset(); - case PAGE_INFO_END_CURSOR_KEYWORD: + case PAGE_INFO_END_CURSOR: return pageValue.getOffset() + ids.size(); - case PAGE_INFO_TOTAL_RECORDS_KEYWORD: + case PAGE_INFO_TOTAL_RECORDS: return pageValue.getPageTotals(); default: break; diff --git a/elide-graphql/src/main/java/com/yahoo/elide/graphql/containers/RootContainer.java b/elide-graphql/src/main/java/com/yahoo/elide/graphql/containers/RootContainer.java index da2780858c..6449e334e7 100644 --- a/elide-graphql/src/main/java/com/yahoo/elide/graphql/containers/RootContainer.java +++ b/elide-graphql/src/main/java/com/yahoo/elide/graphql/containers/RootContainer.java @@ -5,28 +5,24 @@ */ package com.yahoo.elide.graphql.containers; -import com.yahoo.elide.core.EntityDictionary; import com.yahoo.elide.graphql.Environment; import com.yahoo.elide.graphql.PersistentResourceFetcher; -import graphql.language.Field; - /** * Root container for GraphQL requests. */ public class RootContainer implements GraphQLContainer { @Override public Object processFetch(Environment context, PersistentResourceFetcher fetcher) { - EntityDictionary dictionary = context.requestScope.getDictionary(); - Class entityClass = dictionary.getEntityClass(context.field.getName()); - boolean generateTotals = requestContainsPageInfo(context.field); - return fetcher.fetchObject(context, context.requestScope, entityClass, context.ids, - context.sort, context.offset, context.first, context.filters, generateTotals); - } + String entityName = context.field.getName(); + String aliasName = context.field.getAlias(); - public static boolean requestContainsPageInfo(Field field) { - return field.getSelectionSet().getSelections().stream() - .anyMatch(f -> f instanceof Field - && ConnectionContainer.PAGE_INFO_KEYWORD.equals(((Field) f).getName())); + return fetcher.fetchObject( + context.requestScope, + context.requestScope + .getProjectionInfo() + .getProjection(aliasName, entityName), // root-level projection + context.ids + ); } } diff --git a/elide-graphql/src/main/java/com/yahoo/elide/graphql/parser/FragmentResolver.java b/elide-graphql/src/main/java/com/yahoo/elide/graphql/parser/FragmentResolver.java new file mode 100644 index 0000000000..cb00878555 --- /dev/null +++ b/elide-graphql/src/main/java/com/yahoo/elide/graphql/parser/FragmentResolver.java @@ -0,0 +1,139 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ + +package com.yahoo.elide.graphql.parser; + +import com.yahoo.elide.core.exceptions.InvalidEntityBodyException; + +import graphql.language.Document; +import graphql.language.Field; +import graphql.language.FragmentDefinition; +import graphql.language.FragmentSpread; +import graphql.language.Selection; +import graphql.language.SelectionSet; +import java.util.ArrayList; +import java.util.Collections; +import java.util.HashMap; +import java.util.HashSet; +import java.util.List; +import java.util.Map; +import java.util.Set; +import java.util.function.Function; +import java.util.stream.Collectors; +import javax.ws.rs.BadRequestException; + +/** + * Class that fetch {@link FragmentDefinition}s from graphQL {@link Document} and store them for future reference. + */ +public class FragmentResolver { + private final Map fragmentMap = new HashMap<>(); + + public boolean contains(String fragmentName) { + return fragmentMap.containsKey(fragmentName); + } + + public FragmentDefinition get(String fragmentName) { + return fragmentMap.get(fragmentName); + } + + /** + * Fetch fragments from documents. Only fragment definitions would be processed. + * + * @param document graphql document + */ + public void addFragments(Document document) { + addFragments(document.getDefinitions().stream() + .filter(definition -> definition instanceof FragmentDefinition) + .map(definition -> (FragmentDefinition) definition) + .collect(Collectors.toList())); + } + + /** + * Make sure there is not fragment loop in in-coming definitions and store those fragments. + * + * @param fragments fragments to add + */ + private void addFragments(List fragments) { + final Map newFragments = fragments.stream() + .collect(Collectors.toMap(FragmentDefinition::getName, Function.identity())); + + // make sure there is no fragment loop and undefined fragments in fragment definitions + final Set fragmentNames = new HashSet<>(); + fragments.forEach(fragmentDefinition -> validateFragment(newFragments, fragmentDefinition, fragmentNames)); + + this.fragmentMap.putAll(newFragments); + } + + /** + * Recursive DFS to validate that there is not reference loop in a fragment and there is not un-defined + * fragments. + * + * @param fragmentDefinition fragment to be checked + * @param fragmentNames fragment names appear in the current check path + */ + private static void validateFragment( + Map fragmentMap, + FragmentDefinition fragmentDefinition, + Set fragmentNames + ) { + String fragmentName = fragmentDefinition.getName(); + if (fragmentNames.contains(fragmentName)) { + throw new InvalidEntityBodyException("There is a fragment definition loop in: {" + + String.join(",", fragmentNames) + "} with " + fragmentName + " duplicated."); + } + + fragmentNames.add(fragmentName); + + getNestedFragments(fragmentDefinition.getSelectionSet()).stream() + .map(FragmentSpread::getName) + .distinct() + .forEach(name -> { + if (!fragmentMap.containsKey(name)) { + throw new InvalidEntityBodyException(String.format("Unknown fragment {%s}.", name)); + } + validateFragment(fragmentMap, fragmentMap.get(name), fragmentNames); + }); + + fragmentNames.remove(fragmentName); + } + + /** + * Get nested fragments from a selection set, skip other graphQL {@link Field}s. + * This is only for fragment loop validation. + * + * @param selectionSet graphql selection set + * @return nested fragments in the selection set + */ + private static List getNestedFragments(SelectionSet selectionSet) { + return selectionSet.getSelections().stream() + .map(FragmentResolver::getNestedFragments) + .reduce(new ArrayList<>(), (a, b) -> { + a.addAll(b); + return a; + }); + } + + /** + * Get nested fragments from a field, skip other graphQL {@link Field}s. + * This is only for fragment loop validation. + * + * @param selection graphql selection + * @return nested fragments in the selection set of this selection + */ + private static List getNestedFragments(Selection selection) { + if (selection instanceof Field) { + return ((Field) selection).getSelectionSet() == null + || ((Field) selection).getSelectionSet().getSelections().isEmpty() + ? new ArrayList<>() + : getNestedFragments(((Field) selection).getSelectionSet()); + } else if (selection instanceof FragmentSpread) { + return Collections.singletonList((FragmentSpread) selection); + } else { + // TODO: support inline fragment + throw new BadRequestException("Unsupported graphQL selection type: " + selection.getClass()); + } + } +} diff --git a/elide-graphql/src/main/java/com/yahoo/elide/graphql/parser/GraphQLEntityProjectionMaker.java b/elide-graphql/src/main/java/com/yahoo/elide/graphql/parser/GraphQLEntityProjectionMaker.java new file mode 100644 index 0000000000..e0a48a3af6 --- /dev/null +++ b/elide-graphql/src/main/java/com/yahoo/elide/graphql/parser/GraphQLEntityProjectionMaker.java @@ -0,0 +1,557 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ + +package com.yahoo.elide.graphql.parser; + +import static com.yahoo.elide.graphql.KeyWord.EDGES; +import static com.yahoo.elide.graphql.KeyWord.NODE; +import static com.yahoo.elide.graphql.KeyWord.PAGE_INFO; +import static com.yahoo.elide.graphql.KeyWord.PAGE_INFO_TOTAL_RECORDS; +import static com.yahoo.elide.graphql.KeyWord.SCHEMA; +import static com.yahoo.elide.graphql.KeyWord.TYPE; +import static com.yahoo.elide.graphql.KeyWord.TYPENAME; + +import com.yahoo.elide.ElideSettings; +import com.yahoo.elide.core.EntityDictionary; +import com.yahoo.elide.core.RelationshipType; +import com.yahoo.elide.core.exceptions.InvalidEntityBodyException; +import com.yahoo.elide.core.exceptions.InvalidPredicateException; +import com.yahoo.elide.core.exceptions.InvalidValueException; +import com.yahoo.elide.core.filter.dialect.MultipleFilterDialect; +import com.yahoo.elide.core.filter.dialect.ParseException; +import com.yahoo.elide.core.filter.expression.AndFilterExpression; +import com.yahoo.elide.core.filter.expression.FilterExpression; +import com.yahoo.elide.core.pagination.Pagination; +import com.yahoo.elide.core.sort.Sorting; +import com.yahoo.elide.graphql.ModelBuilder; +import com.yahoo.elide.request.Attribute; +import com.yahoo.elide.request.EntityProjection; +import com.yahoo.elide.request.EntityProjection.EntityProjectionBuilder; +import com.yahoo.elide.request.Relationship; + +import graphql.language.Argument; +import graphql.language.Document; +import graphql.language.Field; +import graphql.language.FragmentDefinition; +import graphql.language.FragmentSpread; +import graphql.language.OperationDefinition; +import graphql.language.Selection; +import graphql.language.SelectionSet; +import graphql.language.SourceLocation; +import graphql.parser.Parser; +import lombok.extern.slf4j.Slf4j; +import java.math.BigInteger; +import java.util.Collections; +import java.util.HashMap; +import java.util.List; +import java.util.Map; +import java.util.Optional; +import java.util.stream.Collectors; + +import javax.ws.rs.BadRequestException; +import javax.ws.rs.core.MultivaluedHashMap; + +/** + * This class converts a GraphQL query string into an Elide {@link EntityProjection} using + * {@link #make(String)} method. + */ +@Slf4j +public class GraphQLEntityProjectionMaker { + private final ElideSettings elideSettings; + private final EntityDictionary entityDictionary; + private final MultipleFilterDialect filterDialect; + + private final VariableResolver variableResolver; + private final FragmentResolver fragmentResolver; + + private final Map relationshipMap = new HashMap<>(); + private final Map rootProjections = new HashMap<>(); + private final Map attributeMap = new HashMap<>(); + + /** + * Constructor. + * + * @param elideSettings settings of current Elide instance + * @param variables variables provided in the request + */ + public GraphQLEntityProjectionMaker(ElideSettings elideSettings, Map variables) { + this.elideSettings = elideSettings; + this.entityDictionary = elideSettings.getDictionary(); + this.filterDialect = new MultipleFilterDialect( + elideSettings.getJoinFilterDialects(), + elideSettings.getSubqueryFilterDialects()); + + this.variableResolver = new VariableResolver(variables); + this.fragmentResolver = new FragmentResolver(); + } + + /** + * Constructor. + * + * @param elideSettings settings of current Elide instance + */ + public GraphQLEntityProjectionMaker(ElideSettings elideSettings) { + this(elideSettings, new HashMap<>()); + } + + /** + * Convert a GraphQL query string into a collection of Elide {@link EntityProjection}s. + * + * @param query GraphQL query + * @return all projections in the query + */ + public GraphQLProjectionInfo make(String query) { + Parser parser = new Parser(); + Document parsedDocument; + try { + parsedDocument = parser.parseDocument(query); + } catch (Exception e) { + throw new InvalidEntityBodyException("Can't parse query: " + query); + } + + // resolve fragment definitions + fragmentResolver.addFragments(parsedDocument); + + // resolve operation definitions + parsedDocument.getDefinitions().forEach(definition -> { + if (definition instanceof OperationDefinition) { + // Operations would be converted into EntityProjection tree + OperationDefinition operationDefinition = (OperationDefinition) definition; + if (operationDefinition.getOperation() == OperationDefinition.Operation.SUBSCRIPTION) { + // TODO: support SUBSCRIPTION + return; + } + + // resolve variable definitions in this operation + variableResolver.newScope(operationDefinition); + + addRootProjection(operationDefinition.getSelectionSet()); + } else if (!(definition instanceof FragmentDefinition)) { + throw new InvalidEntityBodyException( + String.format("Unsupported definition type {%s}.", definition.getClass())); + } + }); + + return new GraphQLProjectionInfo(rootProjections, relationshipMap, attributeMap); + } + + /** + * Root projection would be an operation applied on an single entity class. + * The EntityProjection tree would be constructed recursively to add all child projections. + * + * @param selectionSet a root-level selection set + */ + private void addRootProjection(SelectionSet selectionSet) { + List selections = selectionSet.getSelections(); + + selections.stream().forEach(rootSelection -> { + if (!(rootSelection instanceof Field)) { + throw new InvalidEntityBodyException("Entity selection must be a graphQL field."); + } + Field rootSelectionField = (Field) rootSelection; + String entityName = rootSelectionField.getName(); + String aliasName = rootSelectionField.getAlias(); + if (SCHEMA.equals(entityName) || TYPE.equals(entityName)) { + // '__schema' and '__type' would not be handled by entity projection + return; + } + Class entityType = entityDictionary.getEntityClass(rootSelectionField.getName()); + if (entityType == null) { + throw new InvalidEntityBodyException(String.format("Unknown entity {%s}.", + rootSelectionField.getName())); + } + + + String keyName = GraphQLProjectionInfo.computeProjectionKey(aliasName, entityName); + if (rootProjections.containsKey(keyName)) { + throw new InvalidEntityBodyException( + String.format("Found two root level query for Entity {%s} with same alias name", + entityName)); + } + rootProjections.put(keyName, + createProjection(entityType, rootSelectionField)); + }); + + } + + /** + * Construct an {@link EntityProjection} from a GraphQL {@link Field} for an entity type. + * + * @param entityType type of entity to be projected + * @param entityField graphQL field definition + * @return constructed {@link EntityProjection} + */ + private EntityProjection createProjection(Class entityType, Field entityField) { + final EntityProjectionBuilder projectionBuilder = EntityProjection.builder() + .type(entityType); + + entityField.getSelectionSet().getSelections().forEach(selection -> addSelection(selection, projectionBuilder)); + entityField.getArguments().forEach(argument -> addArgument(argument, projectionBuilder)); + + return projectionBuilder.build(); + } + + /** + * Add a graphQL {@link Selection} to an {@link EntityProjection} + * + * @param fieldSelection field/fragment to add + * @param projectionBuilder projection that is being built + */ + private void addSelection(Selection fieldSelection, final EntityProjectionBuilder projectionBuilder) { + if (fieldSelection instanceof FragmentSpread) { + addFragment((FragmentSpread) fieldSelection, projectionBuilder); + } else if (fieldSelection instanceof Field) { + if (EDGES.equals(((Field) fieldSelection).getName()) + || NODE.equals(((Field) fieldSelection).getName())) { + // if this graphql field is 'edges' or 'node', go one level deeper in the graphql document + ((Field) fieldSelection).getSelectionSet().getSelections().forEach( + selection -> addSelection(selection, projectionBuilder)); + } else { + addField((Field) fieldSelection, projectionBuilder); + } + } else { + throw new InvalidEntityBodyException( + String.format("Unsupported selection type {%s}.", fieldSelection.getClass())); + } + } + + /** + * Resolve a graphQL {@link FragmentSpread} into {@link Selection}s and add them to an {@link EntityProjection} + * + * @param fragment graphQL fragment + * @param projectionBuilder projection that is being built + */ + private void addFragment(FragmentSpread fragment, EntityProjectionBuilder projectionBuilder) { + String fragmentName = fragment.getName(); + + FragmentDefinition fragmentDefinition = fragmentResolver.get(fragmentName); + + // type name in type condition of the Fragment must match the entity projection type name + if (entityDictionary.getJsonAliasFor(projectionBuilder.getType()) + .equals(fragmentDefinition.getTypeCondition().getName())) { + fragmentDefinition.getSelectionSet().getSelections() + .forEach(selection -> addSelection(selection, projectionBuilder)); + } + } + + /** + * Add a new graphQL {@link Field} into an {@link EntityProjection} + * + * @param field graphQL field + * @param projectionBuilder projection that is being built + */ + private void addField(Field field, EntityProjectionBuilder projectionBuilder) { + Class parentType = projectionBuilder.getType(); + String fieldName = field.getName(); + + // this field would either be a relationship field or an attribute field + if (entityDictionary.getRelationshipType(parentType, fieldName) != RelationshipType.NONE) { + // handle the case of a relationship field + addRelationship(field, projectionBuilder); + } else if (TYPENAME.equals(fieldName)) { + // '__typename' would not be handled by entityProjection + return; + } else if (PAGE_INFO.equals(fieldName)) { + // only 'totalRecords' needs to be added into the projection's pagination + if (field.getSelectionSet().getSelections().stream() + .anyMatch(selection -> selection instanceof Field + && PAGE_INFO_TOTAL_RECORDS.equals(((Field) selection).getName()))) { + addPageTotal(projectionBuilder); + } + } else { + addAttributeField(field, projectionBuilder); + } + } + + /** + * Create a relationship with projection and add it to the parent projection. + * + * @param relationshipField graphQL field for a relationship + * @param projectionBuilder projection that is being built + */ + private void addRelationship(Field relationshipField, EntityProjectionBuilder projectionBuilder) { + Class parentType = projectionBuilder.getType(); + String relationshipName = relationshipField.getName(); + String relationshipAlias = + relationshipField.getAlias() == null ? relationshipName : relationshipField.getAlias(); + + final Class relationshipType = entityDictionary.getParameterizedType(parentType, relationshipName); + + // build new entity projection with only entity type and entity dictionary + EntityProjection relationshipProjection = createProjection(relationshipType, relationshipField); + Relationship relationship = Relationship.builder() + .name(relationshipName) + .alias(relationshipAlias) + .projection(relationshipProjection) + .build(); + + relationshipMap.put(relationshipField.getSourceLocation(), relationship); + + // add this relationship projection to its parent projection + projectionBuilder.relationship(relationship); + } + + /** + * Add an attribute to an entity projection. + * + * @param attributeField graphQL field for an attribute + * @param projectionBuilder projection that is being built + */ + private void addAttributeField(Field attributeField, EntityProjectionBuilder projectionBuilder) { + Class parentType = projectionBuilder.getType(); + String attributeName = attributeField.getName(); + String attributeAlias = attributeField.getAlias() == null ? attributeName : attributeField.getAlias(); + + Class attributeType = entityDictionary.getType(parentType, attributeName); + if (attributeType != null) { + Attribute attribute = Attribute.builder() + .type(attributeType) + .name(attributeName) + .alias(attributeAlias) + .arguments( + attributeField.getArguments().stream() + .map(graphQLArgument -> com.yahoo.elide.request.Argument.builder() + .name(graphQLArgument.getName()) + .value( + variableResolver.resolveValue( + graphQLArgument.getValue())) + .build()) + .collect(Collectors.toList())) + .build(); + + projectionBuilder.attribute(attribute); + attributeMap.put(attributeField.getSourceLocation(), attribute); + } else { + throw new InvalidEntityBodyException(String.format( + "Unknown attribute field {%s.%s}.", + entityDictionary.getJsonAliasFor(projectionBuilder.getType()), + attributeName)); + } + } + + /** + * Construct Elide {@link Pagination}, {@link Sorting}, {@link Attribute} from GraphQL {@link Argument} and + * add it to the {@link EntityProjection}. + * + * @param argument graphQL argument + * @param projectionBuilder projection that is being built + */ + private void addArgument(Argument argument, EntityProjectionBuilder projectionBuilder) { + String argumentName = argument.getName(); + + if (isPaginationArgument(argumentName)) { + addPagination(argument, projectionBuilder); + } else if (isSortingArgument(argumentName)) { + addSorting(argument, projectionBuilder); + } else if (ModelBuilder.ARGUMENT_FILTER.equals(argumentName)) { + addFilter(argument, projectionBuilder); + } else if (!ModelBuilder.ARGUMENT_OPERATION.equals(argumentName) + && !(ModelBuilder.ARGUMENT_IDS.equals(argumentName)) + && !(ModelBuilder.ARGUMENT_DATA.equals(argumentName))) { + addAttributeArgument(argument, projectionBuilder); + } + } + + /** + * Returns whether or not a GraphQL argument name corresponding to a pagination argument. + * + * @param argumentName Name key of the GraphQL argument + * + * @return {@code true} if the name equals to {@link ModelBuilder#ARGUMENT_FIRST} or + * {@link ModelBuilder#ARGUMENT_AFTER} + */ + private static boolean isPaginationArgument(String argumentName) { + return ModelBuilder.ARGUMENT_FIRST.equals(argumentName) || ModelBuilder.ARGUMENT_AFTER.equals(argumentName); + } + + /** + * Create a {@link Pagination} object from pagination GraphQL argument and attach it to the building + * {@link EntityProjection}. + * + * @param argument graphQL argument + * @param projectionBuilder projection that is being built + */ + private void addPagination(Argument argument, EntityProjectionBuilder projectionBuilder) { + Pagination pagination = projectionBuilder.getPagination() == null + ? Pagination.getDefaultPagination(elideSettings) + : projectionBuilder.getPagination(); + + Object argumentValue = variableResolver.resolveValue(argument.getValue()); + int value = argumentValue instanceof BigInteger + ? ((BigInteger) argumentValue).intValue() + : Integer.parseInt((String) argumentValue); + if (ModelBuilder.ARGUMENT_FIRST.equals(argument.getName())) { + pagination.setLimit(value); + } else if (ModelBuilder.ARGUMENT_AFTER.equals(argument.getName())) { + pagination.setOffset(value); + } + + projectionBuilder.pagination(pagination); + } + + /** + * Make projection return page total records. + * If the projection already has a pagination, use limit and offset from the existing pagination, + * else use the default pagination vales. + * + * @param projectionBuilder projection that is being built + */ + private void addPageTotal(EntityProjectionBuilder projectionBuilder) { + if (projectionBuilder.getPagination() == null) { + Optional pagination = Pagination.fromOffsetAndFirst( + Optional.empty(), + Optional.empty(), + true, + elideSettings + ); + pagination.ifPresent(projectionBuilder::pagination); + } else { + Optional pagination = Pagination.fromOffsetAndFirst( + Optional.of(String.valueOf(projectionBuilder.getPagination().getLimit())), + Optional.of(String.valueOf(projectionBuilder.getPagination().getOffset())), + true, + elideSettings + ); + pagination.ifPresent(projectionBuilder::pagination); + } + } + + /** + * Returns whether or not a GraphQL argument name corresponding to a sorting argument. + * + * @param argumentName Name key of the GraphQL argument + * + * @return {@code true} if the name equals to {@link ModelBuilder#ARGUMENT_SORT} + */ + private static boolean isSortingArgument(String argumentName) { + return ModelBuilder.ARGUMENT_SORT.equals(argumentName); + } + + /** + * Creates a {@link Sorting} object from sorting GraphQL argument value and attaches it to the entity sorted + * according to the newly created {@link Sorting} object. + * + * @param argument An argument that contains the value of sorting spec + * @param projectionBuilder projection that is being built + */ + private void addSorting(Argument argument, EntityProjectionBuilder projectionBuilder) { + String sortRule = (String) variableResolver.resolveValue(argument.getValue()); + Sorting sorting = Sorting.parseSortRule(sortRule); + + // validate sorting rule + try { + sorting.getValidSortingRules(projectionBuilder.getType(), entityDictionary); + } catch (InvalidValueException e) { + throw new BadRequestException("Invalid sorting clause " + sortRule + + " for type " + entityDictionary.getJsonAliasFor(projectionBuilder.getType())); + } + + projectionBuilder.sorting(sorting); + } + + /** + * Add a new filter expression to the entityProjection + * + * @param argument filter argument + * @param projectionBuilder projection that is being built + */ + private void addFilter(Argument argument, EntityProjectionBuilder projectionBuilder) { + FilterExpression filter = buildFilter( + entityDictionary.getJsonAliasFor(projectionBuilder.getType()), + variableResolver.resolveValue(argument.getValue())); + + if (projectionBuilder.getFilterExpression() != null) { + projectionBuilder.filterExpression( + new AndFilterExpression(projectionBuilder.getFilterExpression(), filter)); + } else { + projectionBuilder.filterExpression(filter); + } + } + + /** + * Construct a filter expression from a string + * + * @param typeName class type name to apply this filter + * @param filterString Elide filter in string format + * @return constructed filter expression + */ + private FilterExpression buildFilter(String typeName, Object filterString) { + if (!(filterString instanceof String)) { + throw new BadRequestException("Filter of type " + typeName + " is not StringValue."); + } + + try { + return filterDialect.parseTypedExpression(typeName, toQueryParams(typeName, filterString)).get(typeName); + } catch (ParseException e) { + log.debug("Filter parse exception caught", e); + throw new InvalidPredicateException("Could not parse filter " + filterString + " for type: " + typeName); + } + } + + /** + * Convert a type name and filter string to a map that mimic query params comes from request. + * + * @param typeName class type name to apply this filter + * @param filterString Elide filter in string format + * @return constructed map + */ + private static MultivaluedHashMap toQueryParams(String typeName, Object filterString) { + return new MultivaluedHashMap() { + { + put("filter[" + typeName + "]", Collections.singletonList((String) filterString)); + } + }; + } + + /** + * Add argument for a field/relationship of an entity + * + * @param argument an argument which name should match a field name/alias + * @param projectionBuilder projection that is being built + */ + private void addAttributeArgument(Argument argument, EntityProjectionBuilder projectionBuilder) { + String argumentName = argument.getName(); + Class entityType = projectionBuilder.getType(); + + Attribute existingAttribute = projectionBuilder.getAttributeByAlias(argumentName); + + com.yahoo.elide.request.Argument elideArgument = com.yahoo.elide.request.Argument.builder() + .name(argumentName) + .value(variableResolver.resolveValue(argument.getValue())) + .build(); + + if (existingAttribute != null) { + // add a new argument to the existing attribute + Attribute toAdd = Attribute.builder() + .type(existingAttribute.getType()) + .name(existingAttribute.getName()) + .alias(existingAttribute.getAlias()) + .argument(elideArgument) + .build(); + + projectionBuilder.attribute(toAdd); + } else { + Class attributeType = entityDictionary.getType(entityType, argumentName); + if (attributeType == null) { + throw new InvalidEntityBodyException( + String.format("Invalid attribute field/alias for argument: {%s}.{%s}", + entityType, + argumentName) + ); + } + + // create a new attribute if this attribute doesn't exist in the projection + Attribute toAdd = Attribute.builder() + .type(attributeType) + .name(argumentName) + .alias(argumentName) + .argument(elideArgument) + .build(); + + projectionBuilder.attribute(toAdd); + } + } +} diff --git a/elide-graphql/src/main/java/com/yahoo/elide/graphql/parser/GraphQLProjectionInfo.java b/elide-graphql/src/main/java/com/yahoo/elide/graphql/parser/GraphQLProjectionInfo.java new file mode 100644 index 0000000000..d36d389e33 --- /dev/null +++ b/elide-graphql/src/main/java/com/yahoo/elide/graphql/parser/GraphQLProjectionInfo.java @@ -0,0 +1,37 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ + +package com.yahoo.elide.graphql.parser; + +import com.yahoo.elide.request.Attribute; +import com.yahoo.elide.request.EntityProjection; +import com.yahoo.elide.request.Relationship; +import graphql.language.SourceLocation; +import lombok.AllArgsConstructor; +import lombok.Getter; + +import java.util.Map; + +/** + * A helper class that contains a collection of root-level entity projections and relationship map constructed from + * a {@link graphql.language.Document}. + */ +@AllArgsConstructor +public class GraphQLProjectionInfo { + @Getter private final Map projections; + + @Getter private final Map relationshipMap; + + @Getter private final Map attributeMap; + + public EntityProjection getProjection(String aliasName, String entityName) { + return projections.get(computeProjectionKey(aliasName, entityName)); + } + + public static String computeProjectionKey(String aliasName, String entityName) { + return (aliasName == null ? "" : aliasName) + ":" + entityName; + } +} diff --git a/elide-graphql/src/main/java/com/yahoo/elide/graphql/parser/VariableResolver.java b/elide-graphql/src/main/java/com/yahoo/elide/graphql/parser/VariableResolver.java new file mode 100644 index 0000000000..a2a6e6dc85 --- /dev/null +++ b/elide-graphql/src/main/java/com/yahoo/elide/graphql/parser/VariableResolver.java @@ -0,0 +1,122 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ + +package com.yahoo.elide.graphql.parser; + +import graphql.language.ArrayValue; +import graphql.language.BooleanValue; +import graphql.language.EnumValue; +import graphql.language.FloatValue; +import graphql.language.IntValue; +import graphql.language.NonNullType; +import graphql.language.NullValue; +import graphql.language.ObjectField; +import graphql.language.ObjectValue; +import graphql.language.OperationDefinition; +import graphql.language.StringValue; +import graphql.language.Type; +import graphql.language.Value; +import graphql.language.VariableDefinition; +import graphql.language.VariableReference; + +import java.util.HashMap; +import java.util.Map; +import java.util.stream.Collectors; + +import javax.ws.rs.BadRequestException; + +/** + * Class that contains variables provided in graphql request and can resolve variables based on + * {@link graphql.language.OperationDefinition} scope. + * 1. variables defined in request is global + * 2. variables defined in each operation is operation-scoped + */ +class VariableResolver { + private final Map requestVariables; + private final Map scopeVariables = new HashMap<>(); + + VariableResolver(Map variables) { + this.requestVariables = new HashMap<>(variables); + } + + /** + * Start a new variable scope for operation, clear all variables in the previous scope and add request variables + * into every new scope. + * + * @param operation operation definition + */ + public void newScope(OperationDefinition operation) { + this.scopeVariables.clear(); + this.scopeVariables.putAll(requestVariables); + operation.getVariableDefinitions().forEach(this::addVariable); + } + + /** + * Resolve {@link VariableDefinition} and store result in the variable map. + * We don't need to worry about resolving graphql {@link graphql.language.TypeName} here because Elide-core + * knows the correct type of each field/argument. + * + * @param definition definition to resolve + */ + private void addVariable(VariableDefinition definition) { + Type variableType = definition.getType(); + String variableName = definition.getName(); + Value defaultValue = definition.getDefaultValue(); + + if (defaultValue == null) { + if (variableType instanceof NonNullType && scopeVariables.get(variableName) == null) { + // value of non-null variable must be resolvable + throw new BadRequestException("Undefined non-null variable " + variableName); + } else { + // this would put 'null' for this variable if it is not stored in the map + scopeVariables.put(variableName, scopeVariables.get(variableName)); + } + } else { + if (!scopeVariables.containsKey(variableName)) { + // create a new variable with default value + scopeVariables.put(variableName, resolveValue(defaultValue)); + } + } + } + + /** + * Resolve the real value of a GraphQL {@link Value} object. Use variables in request if necessary. + * + * @param value requested variable value + * @return resolved value of given variable + */ + public Object resolveValue(Value value) { + if (value instanceof BooleanValue) { + return ((BooleanValue) value).isValue(); + } else if (value instanceof EnumValue) { + // TODO + throw new BadRequestException("Enum value is not supported."); + } else if (value instanceof FloatValue) { + return ((FloatValue) value).getValue(); + } else if (value instanceof IntValue) { + return ((IntValue) value).getValue(); + } else if (value instanceof NullValue) { + return null; + } else if (value instanceof StringValue) { + return ((StringValue) value).getValue(); + } else if (value instanceof ObjectValue) { + return ((ObjectValue) value).getObjectFields().stream() + .collect(Collectors.toMap(ObjectField::getName, ObjectField::getValue)); + } else if (value instanceof ArrayValue) { + return ((ArrayValue) value).getValues().stream() + .map(this::resolveValue) + .collect(Collectors.toList()); + } else if (value instanceof VariableReference) { + String variableName = ((VariableReference) value).getName(); + if (!scopeVariables.containsKey(variableName)) { + throw new BadRequestException("Can't resolve variable reference " + variableName); + } + + return scopeVariables.get(variableName); + } + throw new BadRequestException("Unknown variable value type " + value.getClass()); + } +} diff --git a/elide-graphql/src/test/java/com/yahoo/elide/graphql/FetcherDeleteTest.java b/elide-graphql/src/test/java/com/yahoo/elide/graphql/FetcherDeleteTest.java index 6c395c7e6f..7f8146a610 100644 --- a/elide-graphql/src/test/java/com/yahoo/elide/graphql/FetcherDeleteTest.java +++ b/elide-graphql/src/test/java/com/yahoo/elide/graphql/FetcherDeleteTest.java @@ -81,6 +81,17 @@ public void testNestedBadInput() { assertQueryFails(graphQLRequest); } + @Test + public void testBadArgument() { + String graphQLRequest = "mutation { " + + "author(unknown: \"1\") { " + + "books(op:DELETE) { " + + "edges { node { id } } " + + "} " + + "} " + + "}"; + assertParsingFails(graphQLRequest); + } @Test public void testNestedSingleId() throws Exception { diff --git a/elide-graphql/src/test/java/com/yahoo/elide/graphql/FetcherFetchTest.java b/elide-graphql/src/test/java/com/yahoo/elide/graphql/FetcherFetchTest.java index d28092f42c..06aa20e348 100644 --- a/elide-graphql/src/test/java/com/yahoo/elide/graphql/FetcherFetchTest.java +++ b/elide-graphql/src/test/java/com/yahoo/elide/graphql/FetcherFetchTest.java @@ -8,11 +8,10 @@ import static org.junit.jupiter.api.Assertions.assertTrue; -import com.yahoo.elide.core.DataStoreTransaction; -import com.yahoo.elide.core.RequestScope; import org.junit.jupiter.api.Test; import graphql.ExecutionResult; +import java.util.HashMap; /** * Test the Fetch operation. @@ -27,6 +26,11 @@ public void testRootSingle() throws Exception { runComparisonTest("rootSingle"); } + @Test + public void testRootUnknownField() throws Exception { + assertParsingFails(loadGraphQLRequest("fetch/rootUnknownField.graphql")); + } + @Test public void testRootMultipleIds() throws Exception { runComparisonTest("rootMultipleIds"); @@ -52,6 +56,11 @@ public void testRootCollectionSort() throws Exception { runComparisonTest("rootCollectionSort"); } + @Test + public void testRootCollectionInvalidSort() throws Exception { + assertParsingFails(loadGraphQLRequest("fetch/rootCollectionInvalidSort.graphql")); + } + @Test public void testRootCollectionMultiSort() throws Exception { runComparisonTest("rootCollectionMultiSort"); @@ -98,10 +107,7 @@ public void testDateLessThanFilter() throws Exception { } @Test - public void testFailuresWithBody() throws Exception { - DataStoreTransaction tx = inMemoryDataStore.beginTransaction(); - RequestScope requestScope = new GraphQLRequestScope(tx, null, settings); - + public void testFailuresWithBody() { String graphQLRequest = "{ " + "book(ids: [\"1\"], data: [{\"id\": \"1\"}]) { " + "edges { node { " @@ -110,8 +116,8 @@ public void testFailuresWithBody() throws Exception { + "}}" + "} " + "}"; - ExecutionResult result = api.execute(graphQLRequest, requestScope); - assertTrue(!result.getErrors().isEmpty()); + + assertParsingFails(graphQLRequest); } @Test @@ -145,32 +151,82 @@ public void testComputedAttributes() throws Exception { } @Test - public void testFetchWithFragments() throws Exception { - runComparisonTest("fetchWithFragment"); + public void testFragmentCorrect() throws Exception { + runComparisonTest("fragmentCorrect"); + } + + @Test + public void testFragmentLoop() throws Exception { + assertParsingFails(loadGraphQLRequest("fetch/fragmentLoop.graphql")); + } + + @Test + public void testFragmentInline() throws Exception { + assertParsingFails(loadGraphQLRequest("fetch/fragmentInline.graphql")); + } + + @Test + public void testFragmentUnknown() throws Exception { + assertParsingFails(loadGraphQLRequest("fetch/fragmentUnknown.graphql")); + } + + @Test + public void testVariableDefinition() throws Exception { + runComparisonTest("variableDefinition"); + } + + @Test + public void testVariableUnknownReference() throws Exception { + assertParsingFails(loadGraphQLRequest("fetch/variableUnknownReference.graphql")); } @Test - public void testSchemaIntrospection() throws Exception { - DataStoreTransaction tx = inMemoryDataStore.beginTransaction(); - RequestScope requestScope = new GraphQLRequestScope(tx, null, settings); + public void testVariableInvalidNonNull() throws Exception { + assertParsingFails(loadGraphQLRequest("fetch/variableInvalidNonNull.graphql")); + } + @Test + public void testAliasAttribute() throws Exception { + runComparisonTest("aliasAttribute"); + } + + @Test + public void testAliasRelationship() throws Exception { + runComparisonTest("aliasRelationship"); + } + + @Test + public void testAliasSameRelationship() throws Exception { + runComparisonTest("aliasSameRelationship"); + } + + @Test + public void testAliasAmbiguous() throws Exception { + assertParsingFails(loadGraphQLRequest("fetch/aliasAmbiguous.graphql")); + } + + @Test + public void testAliasPartialQueryAmbiguous() throws Exception { + assertParsingFails(loadGraphQLRequest("fetch/aliasPartialQueryAmbiguous.graphql")); + } + + @Test + public void testSchemaIntrospection() { String graphQLRequest = "{" - + "__schema {" - + "types {" - + " name" - + "}" - + "}" - + "}"; - ExecutionResult result = api.execute(graphQLRequest, requestScope); + + "__schema {" + + "types {" + + " name" + + "}" + + "}" + + "}"; + + ExecutionResult result = runGraphQLRequest(graphQLRequest, new HashMap<>()); assertTrue(result.getErrors().isEmpty()); } @Test - public void testTypeIntrospection() throws Exception { - DataStoreTransaction tx = inMemoryDataStore.beginTransaction(); - RequestScope requestScope = new GraphQLRequestScope(tx, null, settings); - + public void testTypeIntrospection() { String graphQLRequest = "{" + "__type(name: \"author\") {" + " name" @@ -179,7 +235,7 @@ public void testTypeIntrospection() throws Exception { + " }" + "}" + "}"; - ExecutionResult result = api.execute(graphQLRequest, requestScope); + ExecutionResult result = runGraphQLRequest(graphQLRequest, new HashMap<>()); assertTrue(result.getErrors().isEmpty()); } diff --git a/elide-graphql/src/test/java/com/yahoo/elide/graphql/FetcherRemoveTest.java b/elide-graphql/src/test/java/com/yahoo/elide/graphql/FetcherRemoveTest.java index 2a1a757435..c4adac8d66 100644 --- a/elide-graphql/src/test/java/com/yahoo/elide/graphql/FetcherRemoveTest.java +++ b/elide-graphql/src/test/java/com/yahoo/elide/graphql/FetcherRemoveTest.java @@ -54,6 +54,18 @@ public void testNestedBadInput() { assertQueryFails(graphQLRequest); } + @Test + public void testBadArgument() { + String graphQLRequest = "mutation { " + + "author(unknown: \"1\") { " + + "books(op:REMOVE) { " + + "edges { node { id } } " + + "} " + + "} " + + "}"; + assertParsingFails(graphQLRequest); + } + @Test public void testRootCollection() throws Exception { // Part 1: Delete the objects diff --git a/elide-graphql/src/test/java/com/yahoo/elide/graphql/FetcherReplaceTest.java b/elide-graphql/src/test/java/com/yahoo/elide/graphql/FetcherReplaceTest.java index ef27346e5e..326e65ef2e 100644 --- a/elide-graphql/src/test/java/com/yahoo/elide/graphql/FetcherReplaceTest.java +++ b/elide-graphql/src/test/java/com/yahoo/elide/graphql/FetcherReplaceTest.java @@ -5,7 +5,6 @@ */ package com.yahoo.elide.graphql; -import org.junit.jupiter.api.Disabled; import org.junit.jupiter.api.Test; public class FetcherReplaceTest extends PersistentResourceFetcherTest { @@ -25,11 +24,9 @@ public void testReplaceEmptyCollections() throws Exception { runComparisonTest("replaceEmptyCollections"); } - // FIXME: Remove stack traces from error handler... - @Disabled + @Test public void testReplaceWithIdsFails() throws Exception { - String expectedMessage = "Exception while fetching data: javax.ws.rs.BadRequestException: REPLACE " - + "must not include ids argument"; + String expectedMessage = "Exception while fetching data (/book) : REPLACE must not include ids argument"; runErrorComparisonTest("replaceWithIdsFails", expectedMessage); } diff --git a/elide-graphql/src/test/java/com/yahoo/elide/graphql/GraphQLEndpointTest.java b/elide-graphql/src/test/java/com/yahoo/elide/graphql/GraphQLEndpointTest.java index 4138940153..396012cf1e 100644 --- a/elide-graphql/src/test/java/com/yahoo/elide/graphql/GraphQLEndpointTest.java +++ b/elide-graphql/src/test/java/com/yahoo/elide/graphql/GraphQLEndpointTest.java @@ -337,22 +337,8 @@ void testPartialResponse() throws IOException, JSONException { ) ).toQuery(); - String expectedData = document( - selection( - field( - "book", - selections( - field("user1SecretField", "null", false), - field("id", "1"), - field("title", "My first book") - ) - ) - ) - ).toResponse(); - Response response = endpoint.post(user2, graphQLRequestToJSON(graphQLRequest)); assertHasErrors(response); - assert200DataEqual(response, expectedData); } @Test @@ -791,6 +777,233 @@ void testQueryAMapWithBadFields() throws IOException { assertHasErrors(response); } + + @Test + public void testMultipleRoot() throws JSONException { + String graphQLRequest = document( + selections( + field( + "author", + selections( + field("id"), + field("name"), + field( + "books", + selection( + field("title") + ) + ) + ) + ), + field( + "book", + selections( + field("id"), + field("title"), + field( + "authors", + selection( + field("name") + ) + ) + ) + ) + ) + ).toQuery(); + + String graphQLResponse = document( + selections( + field( + "author", + selections( + field("id", "1"), + field("name", "Ricky Carmichael"), + field( + "books", + selections( + field("title", "My first book") + ) + ) + ), + selections( + field("id", "2"), + field("name", "The Silent Author"), + field( + "books", "", false + ) + ) + ), + field( + "book", + selections( + field("id", "1"), + field("title", "My first book"), + field( + "authors", + selection( + field("name", "Ricky Carmichael") + ) + ) + ) + ) + ) + ).toResponse(); + + + Response response = endpoint.post(user1, graphQLRequestToJSON(graphQLRequest)); + assert200EqualBody(response, graphQLResponse); + } + + @Test + public void testMultipleQueryWithAlias() throws JSONException { + String graphQLRequest = document( + selections( + field( + "AuthorBook", + "author", + selections( + field("id"), + field( + "books", + selection( + field("title") + ) + ) + ) + ), + field( + "AuthorName", + "author", + selections( + field("id"), + field("name") + ) + ) + ) + ).toQuery(); + String graphQLResponse = document( + selections( + field( + "AuthorBook", + selections( + field("id", "1"), + field( + "books", + selections( + field("title", "My first book") + ) + ) + ), + selections( + field("id", "2"), + field( + "books", "", false + ) + ) + ), + field( + "AuthorName", + selections( + field("id", "1"), + field("name", "Ricky Carmichael") + ), + selections( + field("id", "2"), + field("name", "The Silent Author") + ) + ) + ) + ).toResponse(); + + + Response response = endpoint.post(user1, graphQLRequestToJSON(graphQLRequest)); + assert200EqualBody(response, graphQLResponse); + } + + @Test + public void testMultipleQueryWithAliasAndArguments() throws JSONException { + String graphQLRequest = document( + query( + "myQuery", + variableDefinitions( + variableDefinition("author1", "[String]"), + variableDefinition("author2", "[String]") + ), + selections( + field( + "Author_1", + "author", + arguments( + argument("ids", "$author1") + ), + selections( + field("id"), + field("name"), + field( + "books", + selection( + field("title") + ) + ) + ) + ), + field( + "Author_2", + "author", + arguments( + argument("ids", "$author2") + ), + selections( + field("id"), + field("name"), + field( + "books", + selection( + field("title") + ) + ) + ) + ) + ) + ) + ).toQuery(); + String graphQLResponse = document( + selections( + field( + "Author_1", + selections( + field("id", "1"), + field("name", "Ricky Carmichael"), + field( + "books", + selections( + field("title", "My first book") + ) + ) + ) + ), + field( + "Author_2", + selections( + field("id", "2"), + field("name", "The Silent Author"), + field( + "books", "", false + ) + ) + ) + ) + ).toResponse(); + + + Map variables = new HashMap<>(); + variables.put("author1", "1"); + variables.put("author2", "2"); + + Response response = endpoint.post(user1, graphQLRequestToJSON(graphQLRequest, variables)); + assert200EqualBody(response, graphQLResponse); + } + private static String graphQLRequestToJSON(String request) { return graphQLRequestToJSON(request, new HashMap<>()); } diff --git a/elide-graphql/src/test/java/com/yahoo/elide/graphql/GraphQLTest.java b/elide-graphql/src/test/java/com/yahoo/elide/graphql/GraphQLTest.java index d3da3f6031..417b4678ab 100644 --- a/elide-graphql/src/test/java/com/yahoo/elide/graphql/GraphQLTest.java +++ b/elide-graphql/src/test/java/com/yahoo/elide/graphql/GraphQLTest.java @@ -21,7 +21,7 @@ /** * Bootstrap for GraphQL tests. */ -public class GraphQLTest { +public abstract class GraphQLTest { protected EntityDictionary dictionary; public GraphQLTest() { diff --git a/elide-graphql/src/test/java/com/yahoo/elide/graphql/ModelBuilderTest.java b/elide-graphql/src/test/java/com/yahoo/elide/graphql/ModelBuilderTest.java index 02d381ce15..2f80c76c29 100644 --- a/elide-graphql/src/test/java/com/yahoo/elide/graphql/ModelBuilderTest.java +++ b/elide-graphql/src/test/java/com/yahoo/elide/graphql/ModelBuilderTest.java @@ -7,13 +7,16 @@ package com.yahoo.elide.graphql; import static graphql.Assert.assertNotNull; +import static org.junit.jupiter.api.Assertions.assertEquals; import static org.junit.jupiter.api.Assertions.assertFalse; import static org.junit.jupiter.api.Assertions.assertNotEquals; import static org.junit.jupiter.api.Assertions.assertNull; import static org.junit.jupiter.api.Assertions.assertTrue; import static org.mockito.Mockito.mock; +import com.yahoo.elide.core.ArgumentType; import com.yahoo.elide.core.EntityDictionary; +import com.yahoo.elide.core.sort.Sorting; import example.Author; import example.Book; @@ -31,6 +34,7 @@ import graphql.schema.GraphQLSchema; import java.util.Collections; +import java.util.HashSet; import java.util.Set; import java.util.stream.Collectors; @@ -189,6 +193,24 @@ public void testBuild() { assertTrue(booksInputType.getWrappedType().equals(bookInputType)); } + @Test + public void checkAttributeArguments() { + Set arguments = new HashSet<>(); + arguments.add(new ArgumentType(SORT, Sorting.SortOrder.class)); + arguments.add(new ArgumentType(TYPE, String.class)); + dictionary.addArgumentsToAttribute(Book.class, PUBLISH_DATE, arguments); + + DataFetcher fetcher = mock(DataFetcher.class); + ModelBuilder builder = new ModelBuilder(dictionary, fetcher); + + GraphQLSchema schema = builder.build(); + + GraphQLObjectType bookType = getConnectedType((GraphQLObjectType) schema.getType(BOOK), null); + assertEquals(2, bookType.getFieldDefinition(PUBLISH_DATE).getArguments().size()); + assertTrue(bookType.getFieldDefinition(PUBLISH_DATE).getArgument(SORT).getType() instanceof GraphQLEnumType); + assertTrue(bookType.getFieldDefinition(PUBLISH_DATE).getArgument(TYPE).getType().equals(Scalars.GraphQLString)); + } + private GraphQLObjectType getConnectedType(GraphQLObjectType root, String connectionName) { GraphQLList edgesType = (GraphQLList) root.getFieldDefinition(EDGES).getType(); GraphQLObjectType rootType = (GraphQLObjectType) diff --git a/elide-graphql/src/test/java/com/yahoo/elide/graphql/PersistentResourceFetcherTest.java b/elide-graphql/src/test/java/com/yahoo/elide/graphql/PersistentResourceFetcherTest.java index a1bfd44d71..eb55bccb9b 100644 --- a/elide-graphql/src/test/java/com/yahoo/elide/graphql/PersistentResourceFetcherTest.java +++ b/elide-graphql/src/test/java/com/yahoo/elide/graphql/PersistentResourceFetcherTest.java @@ -7,25 +7,25 @@ import static org.junit.jupiter.api.Assertions.assertEquals; import static org.junit.jupiter.api.Assertions.assertNotEquals; +import static org.junit.jupiter.api.Assertions.assertThrows; import static org.junit.jupiter.api.Assertions.fail; import com.yahoo.elide.ElideSettings; import com.yahoo.elide.ElideSettingsBuilder; import com.yahoo.elide.core.DataStoreTransaction; -import com.yahoo.elide.core.RequestScope; import com.yahoo.elide.core.datastore.inmemory.HashMapDataStore; import com.yahoo.elide.core.datastore.inmemory.InMemoryDataStore; import com.yahoo.elide.core.filter.dialect.RSQLFilterDialect; +import com.yahoo.elide.graphql.parser.GraphQLEntityProjectionMaker; +import com.yahoo.elide.graphql.parser.GraphQLProjectionInfo; import com.yahoo.elide.utils.coerce.CoerceUtil; import com.fasterxml.jackson.core.JsonProcessingException; import com.fasterxml.jackson.databind.ObjectMapper; - import example.Author; import example.Book; import example.Pseudonym; import example.Publisher; - import org.apache.tools.ant.util.FileUtils; import org.junit.jupiter.api.AfterEach; import org.junit.jupiter.api.BeforeEach; @@ -36,7 +36,6 @@ import graphql.ExecutionResult; import graphql.GraphQL; import graphql.GraphQLError; - import java.io.IOException; import java.io.InputStream; import java.io.InputStreamReader; @@ -44,6 +43,7 @@ import java.util.Arrays; import java.util.Collections; import java.util.Date; +import java.util.HashMap; import java.util.HashSet; import java.util.List; import java.util.Map; @@ -85,7 +85,7 @@ public PersistentResourceFetcherTest() { inMemoryDataStore.populateEntityDictionary(dictionary); - ModelBuilder builder = new ModelBuilder(dictionary, new PersistentResourceFetcher(settings)); + ModelBuilder builder = new ModelBuilder(dictionary, new PersistentResourceFetcher()); api = new GraphQL(builder.build()); @@ -171,7 +171,9 @@ protected void assertQueryEquals(String graphQLRequest, String expectedResponse, boolean isMutation = graphQLRequest.startsWith("mutation"); DataStoreTransaction tx = inMemoryDataStore.beginTransaction(); - RequestScope requestScope = new GraphQLRequestScope(tx, null, settings); + GraphQLProjectionInfo projectionInfo = + new GraphQLEntityProjectionMaker(settings, variables).make(graphQLRequest); + GraphQLRequestScope requestScope = new GraphQLRequestScope(tx, null, settings, projectionInfo); ExecutionResult result = api.execute(graphQLRequest, requestScope, variables); // NOTE: We're forcing commit even in case of failures. GraphQLEndpoint tests should ensure we do not commit on @@ -196,7 +198,8 @@ protected void assertQueryFailsWith(String graphQLRequest, String expectedMessag boolean isMutation = graphQLRequest.startsWith("mutation"); DataStoreTransaction tx = inMemoryDataStore.beginTransaction(); - RequestScope requestScope = new GraphQLRequestScope(tx, null, settings); + GraphQLProjectionInfo projectionInfo = new GraphQLEntityProjectionMaker(settings).make(graphQLRequest); + GraphQLRequestScope requestScope = new GraphQLRequestScope(tx, null, settings, projectionInfo); ExecutionResult result = api.execute(graphQLRequest, requestScope); if (isMutation) { @@ -214,10 +217,7 @@ protected void assertQueryFailsWith(String graphQLRequest, String expectedMessag } protected void assertQueryFails(String graphQLRequest) { - DataStoreTransaction tx = inMemoryDataStore.beginTransaction(); - RequestScope requestScope = new GraphQLRequestScope(tx, null, settings); - - ExecutionResult result = api.execute(graphQLRequest, requestScope); + ExecutionResult result = runGraphQLRequest(graphQLRequest, new HashMap<>()); //debug for errors LOG.debug("Errors = [" + errorsToString(result.getErrors()) + "]"); @@ -225,6 +225,18 @@ protected void assertQueryFails(String graphQLRequest) { assertNotEquals(result.getErrors().size(), 0); } + protected void assertParsingFails(String graphQLRequest) { + assertThrows(Exception.class, () -> new GraphQLEntityProjectionMaker(settings).make(graphQLRequest)); + } + + protected ExecutionResult runGraphQLRequest(String graphQLRequest, Map variables) { + DataStoreTransaction tx = inMemoryDataStore.beginTransaction(); + GraphQLProjectionInfo projectionInfo = new GraphQLEntityProjectionMaker(settings).make(graphQLRequest); + GraphQLRequestScope requestScope = new GraphQLRequestScope(tx, null, settings, projectionInfo); + + return api.execute(graphQLRequest, requestScope, variables); + } + protected String errorsToString(List errors) { return errors.stream() .map(GraphQLError::getMessage) diff --git a/elide-graphql/src/test/java/graphqlEndpointTestModels/Author.java b/elide-graphql/src/test/java/graphqlEndpointTestModels/Author.java index e78d7ae535..75b21f4f19 100644 --- a/elide-graphql/src/test/java/graphqlEndpointTestModels/Author.java +++ b/elide-graphql/src/test/java/graphqlEndpointTestModels/Author.java @@ -28,7 +28,7 @@ import javax.persistence.OneToOne; import javax.persistence.Transient; -@Include +@Include(rootLevel = true) @Entity @CreatePermission(expression = Author.PERMISSION) @ReadPermission(expression = Author.PERMISSION) diff --git a/elide-graphql/src/test/resources/graphql/requests/fetch/aliasAmbiguous.graphql b/elide-graphql/src/test/resources/graphql/requests/fetch/aliasAmbiguous.graphql new file mode 100644 index 0000000000..2b9517d791 --- /dev/null +++ b/elide-graphql/src/test/resources/graphql/requests/fetch/aliasAmbiguous.graphql @@ -0,0 +1,10 @@ +{ + book(ids: ["1"]) { + edges { + node { + alias: id + alias: title + } + } + } +} diff --git a/elide-graphql/src/test/resources/graphql/requests/fetch/aliasAttribute.graphql b/elide-graphql/src/test/resources/graphql/requests/fetch/aliasAttribute.graphql new file mode 100644 index 0000000000..2949278f50 --- /dev/null +++ b/elide-graphql/src/test/resources/graphql/requests/fetch/aliasAttribute.graphql @@ -0,0 +1,9 @@ +{ + book(ids: ["1"]) { + edges { + node { + id_alias: id + } + } + } +} diff --git a/elide-graphql/src/test/resources/graphql/requests/fetch/aliasPartialQueryAmbiguous.graphql b/elide-graphql/src/test/resources/graphql/requests/fetch/aliasPartialQueryAmbiguous.graphql new file mode 100644 index 0000000000..b11f611aa7 --- /dev/null +++ b/elide-graphql/src/test/resources/graphql/requests/fetch/aliasPartialQueryAmbiguous.graphql @@ -0,0 +1,23 @@ +{ + book(ids: ["1"]) { + edges { + node { + alias: id + } + } + } + book(ids: ["1"]) { + edges { + node { + alias: authors { + edges { + node { + id + name + } + } + } + } + } + } +} diff --git a/elide-graphql/src/test/resources/graphql/requests/fetch/aliasRelationship.graphql b/elide-graphql/src/test/resources/graphql/requests/fetch/aliasRelationship.graphql new file mode 100644 index 0000000000..742c5d8178 --- /dev/null +++ b/elide-graphql/src/test/resources/graphql/requests/fetch/aliasRelationship.graphql @@ -0,0 +1,18 @@ +{ + book(ids: ["1"]) { + edges { + node { + id + title + authors_alias: authors { + edges { + node { + id + name + } + } + } + } + } + } +} diff --git a/elide-graphql/src/test/resources/graphql/requests/fetch/aliasSameRelationship.graphql b/elide-graphql/src/test/resources/graphql/requests/fetch/aliasSameRelationship.graphql new file mode 100644 index 0000000000..b7b6c5622a --- /dev/null +++ b/elide-graphql/src/test/resources/graphql/requests/fetch/aliasSameRelationship.graphql @@ -0,0 +1,26 @@ +{ + book(ids: ["1"]) { + edges { + node { + id + title + authors { + edges { + node { + id + name + } + } + } + authors_alias: authors { + edges { + node { + id + name + } + } + } + } + } + } +} diff --git a/elide-graphql/src/test/resources/graphql/requests/fetch/fetchWithFragment.graphql b/elide-graphql/src/test/resources/graphql/requests/fetch/fragmentCorrect.graphql similarity index 100% rename from elide-graphql/src/test/resources/graphql/requests/fetch/fetchWithFragment.graphql rename to elide-graphql/src/test/resources/graphql/requests/fetch/fragmentCorrect.graphql diff --git a/elide-graphql/src/test/resources/graphql/requests/fetch/fragmentInline.graphql b/elide-graphql/src/test/resources/graphql/requests/fetch/fragmentInline.graphql new file mode 100644 index 0000000000..b0d1d28833 --- /dev/null +++ b/elide-graphql/src/test/resources/graphql/requests/fetch/fragmentInline.graphql @@ -0,0 +1,26 @@ +query namedQuery { + author { + ...AuthorFields + } +} + +fragment AuthorFields on author { + edges { + node { + id + books { + ... on book { + edges { + node { + id + title + } + } + } + __typename + } + __typename + } + __typename + } +} diff --git a/elide-graphql/src/test/resources/graphql/requests/fetch/fragmentLoop.graphql b/elide-graphql/src/test/resources/graphql/requests/fetch/fragmentLoop.graphql new file mode 100644 index 0000000000..b0b40e694e --- /dev/null +++ b/elide-graphql/src/test/resources/graphql/requests/fetch/fragmentLoop.graphql @@ -0,0 +1,31 @@ +query namedQuery { + author { + ...AuthorFields + } +} + +fragment AuthorFields on author { + edges { + node { + id + books { + ...BookFields + __typename + } + __typename + } + __typename + } +} + +fragment BookFields on book { + edges { + node { + id + title + authors { + ...AuthorFields + } + } + } +} diff --git a/elide-graphql/src/test/resources/graphql/requests/fetch/fragmentUnknown.graphql b/elide-graphql/src/test/resources/graphql/requests/fetch/fragmentUnknown.graphql new file mode 100644 index 0000000000..5783bf0ca9 --- /dev/null +++ b/elide-graphql/src/test/resources/graphql/requests/fetch/fragmentUnknown.graphql @@ -0,0 +1,31 @@ +query namedQuery { + author { + ...AuthorFields + } +} + +fragment AuthorFields on author { + edges { + node { + id + books { + ...UnknownFields + __typename + } + __typename + } + __typename + } +} + +fragment BookFields on book { + edges { + node { + id + title + authors { + ...AuthorFields + } + } + } +} diff --git a/elide-graphql/src/test/resources/graphql/requests/fetch/nestedCollectionFilter.graphql b/elide-graphql/src/test/resources/graphql/requests/fetch/nestedCollectionFilter.graphql index cceb5ee145..a1a115b58c 100644 --- a/elide-graphql/src/test/resources/graphql/requests/fetch/nestedCollectionFilter.graphql +++ b/elide-graphql/src/test/resources/graphql/requests/fetch/nestedCollectionFilter.graphql @@ -1,8 +1,8 @@ -{ +query nestedCollectionFilter($Filter: String = "title==\"Libro U*\"") { author(ids: ["1"]) { edges { node { - books(filter: "title==\"Libro U*\"") { + books(filter: $Filter) { edges { node { id diff --git a/elide-graphql/src/test/resources/graphql/requests/fetch/rootCollectionInvalidSort.graphql b/elide-graphql/src/test/resources/graphql/requests/fetch/rootCollectionInvalidSort.graphql new file mode 100644 index 0000000000..68eae41272 --- /dev/null +++ b/elide-graphql/src/test/resources/graphql/requests/fetch/rootCollectionInvalidSort.graphql @@ -0,0 +1,10 @@ +{ + book(sort: "-title|unknown") { + edges { + node { + id + title + } + } + } +} diff --git a/elide-graphql/src/test/resources/graphql/requests/fetch/rootMultiple.graphql b/elide-graphql/src/test/resources/graphql/requests/fetch/rootMultiple.graphql new file mode 100644 index 0000000000..c98de93477 --- /dev/null +++ b/elide-graphql/src/test/resources/graphql/requests/fetch/rootMultiple.graphql @@ -0,0 +1,16 @@ +{ + book { + edges { + node { + id + } + } + } + author { + edges { + node { + id + } + } + } +} diff --git a/elide-graphql/src/test/resources/graphql/requests/fetch/rootUnknownField.graphql b/elide-graphql/src/test/resources/graphql/requests/fetch/rootUnknownField.graphql new file mode 100644 index 0000000000..5a84ed8bd0 --- /dev/null +++ b/elide-graphql/src/test/resources/graphql/requests/fetch/rootUnknownField.graphql @@ -0,0 +1,18 @@ +{ + book(ids: ["1"]) { + edges { + node { + id + unknown + authors { + edges { + node { + id + name + } + } + } + } + } + } +} diff --git a/elide-graphql/src/test/resources/graphql/requests/fetch/variableDefinition.graphql b/elide-graphql/src/test/resources/graphql/requests/fetch/variableDefinition.graphql new file mode 100644 index 0000000000..fdcf2e461c --- /dev/null +++ b/elide-graphql/src/test/resources/graphql/requests/fetch/variableDefinition.graphql @@ -0,0 +1,18 @@ +query testBook($ids : [String] = ["1"]) { + book(ids: $ids) { + edges { + node { + id + title + authors { + edges { + node { + id + name + } + } + } + } + } + } +} diff --git a/elide-graphql/src/test/resources/graphql/requests/fetch/variableInvalidNonNull.graphql b/elide-graphql/src/test/resources/graphql/requests/fetch/variableInvalidNonNull.graphql new file mode 100644 index 0000000000..3143220866 --- /dev/null +++ b/elide-graphql/src/test/resources/graphql/requests/fetch/variableInvalidNonNull.graphql @@ -0,0 +1,18 @@ +query testBook($ids : [String]!) { + book(ids: $ids) { + edges { + node { + id + title + authors { + edges { + node { + id + name + } + } + } + } + } + } +} diff --git a/elide-graphql/src/test/resources/graphql/requests/fetch/variableUnknownReference.graphql b/elide-graphql/src/test/resources/graphql/requests/fetch/variableUnknownReference.graphql new file mode 100644 index 0000000000..f7597e6fee --- /dev/null +++ b/elide-graphql/src/test/resources/graphql/requests/fetch/variableUnknownReference.graphql @@ -0,0 +1,16 @@ +{ + author(ids: ["1"]) { + edges { + node { + books(filter: $UnknownFilter) { + edges { + node { + id + title + } + } + } + } + } + } +} diff --git a/elide-graphql/src/test/resources/graphql/requests/replace/replaceWithidsFails.graphql b/elide-graphql/src/test/resources/graphql/requests/replace/replaceWithIdsFails.graphql similarity index 100% rename from elide-graphql/src/test/resources/graphql/requests/replace/replaceWithidsFails.graphql rename to elide-graphql/src/test/resources/graphql/requests/replace/replaceWithIdsFails.graphql diff --git a/elide-graphql/src/test/resources/graphql/responses/fetch/aliasAttribute.json b/elide-graphql/src/test/resources/graphql/responses/fetch/aliasAttribute.json new file mode 100644 index 0000000000..5c6d4a43c9 --- /dev/null +++ b/elide-graphql/src/test/resources/graphql/responses/fetch/aliasAttribute.json @@ -0,0 +1,11 @@ +{ + "book": { + "edges": [ + { + "node": { + "id_alias": "1" + } + } + ] + } +} diff --git a/elide-graphql/src/test/resources/graphql/responses/fetch/aliasRelationship.json b/elide-graphql/src/test/resources/graphql/responses/fetch/aliasRelationship.json new file mode 100644 index 0000000000..d4187ec800 --- /dev/null +++ b/elide-graphql/src/test/resources/graphql/responses/fetch/aliasRelationship.json @@ -0,0 +1,22 @@ +{ + "book": { + "edges": [ + { + "node": { + "id": "1", + "title": "Libro Uno", + "authors_alias": { + "edges": [ + { + "node": { + "id": "1", + "name": "Mark Twain" + } + } + ] + } + } + } + ] + } +} diff --git a/elide-graphql/src/test/resources/graphql/responses/fetch/aliasSameRelationship.json b/elide-graphql/src/test/resources/graphql/responses/fetch/aliasSameRelationship.json new file mode 100644 index 0000000000..b29162a650 --- /dev/null +++ b/elide-graphql/src/test/resources/graphql/responses/fetch/aliasSameRelationship.json @@ -0,0 +1,32 @@ +{ + "book": { + "edges": [ + { + "node": { + "id": "1", + "title": "Libro Uno", + "authors": { + "edges": [ + { + "node": { + "id": "1", + "name": "Mark Twain" + } + } + ] + }, + "authors_alias": { + "edges": [ + { + "node": { + "id": "1", + "name": "Mark Twain" + } + } + ] + } + } + } + ] + } +} diff --git a/elide-graphql/src/test/resources/graphql/responses/fetch/fetchWithFragment.json b/elide-graphql/src/test/resources/graphql/responses/fetch/fragmentCorrect.json similarity index 100% rename from elide-graphql/src/test/resources/graphql/responses/fetch/fetchWithFragment.json rename to elide-graphql/src/test/resources/graphql/responses/fetch/fragmentCorrect.json diff --git a/elide-graphql/src/test/resources/graphql/responses/fetch/variableDefinition.json b/elide-graphql/src/test/resources/graphql/responses/fetch/variableDefinition.json new file mode 100644 index 0000000000..7138cfd022 --- /dev/null +++ b/elide-graphql/src/test/resources/graphql/responses/fetch/variableDefinition.json @@ -0,0 +1,22 @@ +{ + "book": { + "edges": [ + { + "node": { + "id": "1", + "title": "Libro Uno", + "authors": { + "edges": [ + { + "node": { + "id": "1", + "name": "Mark Twain" + } + } + ] + } + } + } + ] + } +} diff --git a/elide-integration-tests/pom.xml b/elide-integration-tests/pom.xml index 27c20d6c70..497a138ded 100644 --- a/elide-integration-tests/pom.xml +++ b/elide-integration-tests/pom.xml @@ -13,7 +13,7 @@ com.yahoo.elide elide-parent-pom - 4.6.2-SNAPSHOT + 5.0.0-pr6-SNAPSHOT @@ -25,7 +25,7 @@ - 4.6.2-SNAPSHOT + 5.0.0-pr6-SNAPSHOT diff --git a/elide-integration-tests/src/test/java/com/yahoo/elide/errorEncodingTests/EncodedErrorObjectsIT.java b/elide-integration-tests/src/test/java/com/yahoo/elide/errorEncodingTests/EncodedErrorObjectsIT.java index b97332f7d0..6d3029bea8 100644 --- a/elide-integration-tests/src/test/java/com/yahoo/elide/errorEncodingTests/EncodedErrorObjectsIT.java +++ b/elide-integration-tests/src/test/java/com/yahoo/elide/errorEncodingTests/EncodedErrorObjectsIT.java @@ -144,7 +144,7 @@ public void graphQLMutationError() { @Test public void graphQLFetchError() { String request = jsonParser.getJson("/EncodedErrorResponsesIT/graphQLFetchError.req.json"); - String expected = jsonParser.getJson("/EncodedErrorResponsesIT/graphQLFetchError.json"); + String expected = jsonParser.getJson("/EncodedErrorResponsesIT/graphQLFetchErrorObjectEncoded.json"); given() .contentType(GRAPHQL_CONTENT_TYPE) .accept(GRAPHQL_CONTENT_TYPE) diff --git a/elide-integration-tests/src/test/java/com/yahoo/elide/errorEncodingTests/EncodedErrorResponsesIT.java b/elide-integration-tests/src/test/java/com/yahoo/elide/errorEncodingTests/EncodedErrorResponsesIT.java index 8c61529bb7..c058a137ab 100644 --- a/elide-integration-tests/src/test/java/com/yahoo/elide/errorEncodingTests/EncodedErrorResponsesIT.java +++ b/elide-integration-tests/src/test/java/com/yahoo/elide/errorEncodingTests/EncodedErrorResponsesIT.java @@ -149,7 +149,7 @@ public void graphQLMutationError() { @Test public void graphQLFetchError() { String request = jsonParser.getJson("/EncodedErrorResponsesIT/graphQLFetchError.req.json"); - String expected = jsonParser.getJson("/EncodedErrorResponsesIT/graphQLFetchError.json"); + String expected = jsonParser.getJson("/EncodedErrorResponsesIT/graphQLFetchErrorResponseEncoded.json"); given() .contentType(GRAPHQL_CONTENT_TYPE) .accept(GRAPHQL_CONTENT_TYPE) diff --git a/elide-integration-tests/src/test/java/com/yahoo/elide/errorEncodingTests/VerboseEncodedErrorResponsesIT.java b/elide-integration-tests/src/test/java/com/yahoo/elide/errorEncodingTests/VerboseEncodedErrorResponsesIT.java index a8d5b7b1a5..38ca669385 100644 --- a/elide-integration-tests/src/test/java/com/yahoo/elide/errorEncodingTests/VerboseEncodedErrorResponsesIT.java +++ b/elide-integration-tests/src/test/java/com/yahoo/elide/errorEncodingTests/VerboseEncodedErrorResponsesIT.java @@ -145,18 +145,4 @@ public void graphQLMutationError() { .statusCode(HttpStatus.SC_OK) .body(equalTo(expected)); } - - @Test - public void graphQLFetchError() { - String request = jsonParser.getJson("/EncodedErrorResponsesIT/graphQLFetchError.req.json"); - String expected = jsonParser.getJson("/EncodedErrorResponsesIT/graphQLFetchError.json"); - given() - .contentType(GRAPHQL_CONTENT_TYPE) - .accept(GRAPHQL_CONTENT_TYPE) - .body(request) - .post("/graphQL") - .then() - .statusCode(HttpStatus.SC_OK) - .body(equalTo(expected)); - } } diff --git a/elide-integration-tests/src/test/java/com/yahoo/elide/initialization/AbstractApiResourceInitializer.java b/elide-integration-tests/src/test/java/com/yahoo/elide/initialization/AbstractApiResourceInitializer.java index 9f367bf4eb..91a91fdd32 100644 --- a/elide-integration-tests/src/test/java/com/yahoo/elide/initialization/AbstractApiResourceInitializer.java +++ b/elide-integration-tests/src/test/java/com/yahoo/elide/initialization/AbstractApiResourceInitializer.java @@ -57,6 +57,7 @@ public final void setUpServer() throws Exception { String restassuredPort = System.getProperty("restassured.port", System.getenv("restassured.port")); RestAssured.port = Integer.parseInt(StringUtils.isNotEmpty(restassuredPort) ? restassuredPort : "9999"); + RestAssured.enableLoggingOfRequestAndResponseIfValidationFails(); // embedded jetty server server = new Server(RestAssured.port); @@ -70,12 +71,6 @@ public final void setUpServer() throws Exception { servletHolder.setInitParameter("jersey.config.server.provider.packages", packageName); servletHolder.setInitParameter("javax.ws.rs.Application", resourceConfig); - ServletHolder graphqlServlet = servletContextHandler.addServlet(ServletContainer.class, "/graphQL/*"); - graphqlServlet.setInitOrder(2); - graphqlServlet.setInitParameter("jersey.config.server.provider.packages", - com.yahoo.elide.graphql.GraphQLEndpoint.class.getPackage().getName()); - graphqlServlet.setInitParameter("javax.ws.rs.Application", resourceConfig); - log.debug("...Starting Server..."); server.start(); } diff --git a/elide-integration-tests/src/test/java/com/yahoo/elide/initialization/EncodedErrorResponsesTestBinder.java b/elide-integration-tests/src/test/java/com/yahoo/elide/initialization/EncodedErrorResponsesTestBinder.java index 9d0a3a8199..cc1720f2d7 100644 --- a/elide-integration-tests/src/test/java/com/yahoo/elide/initialization/EncodedErrorResponsesTestBinder.java +++ b/elide-integration-tests/src/test/java/com/yahoo/elide/initialization/EncodedErrorResponsesTestBinder.java @@ -50,11 +50,14 @@ public EncodedErrorResponsesTestBinder(final AuditLogger auditLogger, ServiceLoc @Override protected void configure() { + EntityDictionary dictionary = new EntityDictionary(TestCheckMappings.MAPPINGS, injector::inject); + + bind(dictionary).to(EntityDictionary.class); + // Elide instance bindFactory(new Factory() { @Override public Elide provide() { - EntityDictionary dictionary = new EntityDictionary(TestCheckMappings.MAPPINGS, injector::inject); DefaultFilterDialect defaultFilterStrategy = new DefaultFilterDialect(dictionary); RSQLFilterDialect rsqlFilterStrategy = new RSQLFilterDialect(dictionary); diff --git a/elide-integration-tests/src/test/java/com/yahoo/elide/initialization/ErrorObjectsIntegrationTestApplicationResourceConfig.java b/elide-integration-tests/src/test/java/com/yahoo/elide/initialization/ErrorObjectsIntegrationTestApplicationResourceConfig.java index 0b30ca9edf..fa00775443 100644 --- a/elide-integration-tests/src/test/java/com/yahoo/elide/initialization/ErrorObjectsIntegrationTestApplicationResourceConfig.java +++ b/elide-integration-tests/src/test/java/com/yahoo/elide/initialization/ErrorObjectsIntegrationTestApplicationResourceConfig.java @@ -7,13 +7,18 @@ import com.yahoo.elide.audit.TestAuditLogger; +import org.glassfish.hk2.api.ServiceLocator; import org.glassfish.jersey.server.ResourceConfig; +import javax.inject.Inject; + /** * Resource configuration for error objects integration tests. */ public class ErrorObjectsIntegrationTestApplicationResourceConfig extends ResourceConfig { - public ErrorObjectsIntegrationTestApplicationResourceConfig() { - register(new ErrorObjectsTestBinder(new TestAuditLogger())); + + @Inject + public ErrorObjectsIntegrationTestApplicationResourceConfig(ServiceLocator injector) { + register(new ErrorObjectsTestBinder(new TestAuditLogger(), injector)); } } diff --git a/elide-integration-tests/src/test/java/com/yahoo/elide/initialization/ErrorObjectsTestBinder.java b/elide-integration-tests/src/test/java/com/yahoo/elide/initialization/ErrorObjectsTestBinder.java index 74b4f76d96..658c1267b6 100644 --- a/elide-integration-tests/src/test/java/com/yahoo/elide/initialization/ErrorObjectsTestBinder.java +++ b/elide-integration-tests/src/test/java/com/yahoo/elide/initialization/ErrorObjectsTestBinder.java @@ -19,6 +19,7 @@ import example.TestCheckMappings; import org.glassfish.hk2.api.Factory; +import org.glassfish.hk2.api.ServiceLocator; import org.glassfish.hk2.utilities.binding.AbstractBinder; import java.util.Arrays; @@ -28,18 +29,23 @@ */ public class ErrorObjectsTestBinder extends AbstractBinder { private final AuditLogger auditLogger; + private final ServiceLocator injector; - public ErrorObjectsTestBinder(final AuditLogger auditLogger) { + public ErrorObjectsTestBinder(final AuditLogger auditLogger, ServiceLocator injector) { this.auditLogger = auditLogger; + this.injector = injector; } @Override protected void configure() { + EntityDictionary dictionary = new EntityDictionary(TestCheckMappings.MAPPINGS, injector::inject); + + bind(dictionary).to(EntityDictionary.class); + // Elide instance bindFactory(new Factory() { @Override public Elide provide() { - EntityDictionary dictionary = new EntityDictionary(TestCheckMappings.MAPPINGS); DefaultFilterDialect defaultFilterStrategy = new DefaultFilterDialect(dictionary); RSQLFilterDialect rsqlFilterStrategy = new RSQLFilterDialect(dictionary); diff --git a/elide-integration-tests/src/test/java/com/yahoo/elide/initialization/IntegrationTest.java b/elide-integration-tests/src/test/java/com/yahoo/elide/initialization/IntegrationTest.java index dcd1a0a63e..55ce4ccb9b 100644 --- a/elide-integration-tests/src/test/java/com/yahoo/elide/initialization/IntegrationTest.java +++ b/elide-integration-tests/src/test/java/com/yahoo/elide/initialization/IntegrationTest.java @@ -80,7 +80,7 @@ protected IntegrationTest(final Class resourceConfig, } } - private DataStoreTestHarness createHarness() { + protected DataStoreTestHarness createHarness() { try { final String dataStoreSupplierName = System.getProperty("dataStoreHarness"); diff --git a/elide-integration-tests/src/test/java/com/yahoo/elide/initialization/StandardTestBinder.java b/elide-integration-tests/src/test/java/com/yahoo/elide/initialization/StandardTestBinder.java index 993f3a2f95..c308c37954 100644 --- a/elide-integration-tests/src/test/java/com/yahoo/elide/initialization/StandardTestBinder.java +++ b/elide-integration-tests/src/test/java/com/yahoo/elide/initialization/StandardTestBinder.java @@ -24,6 +24,7 @@ import org.glassfish.hk2.utilities.binding.AbstractBinder; import java.util.Arrays; +import java.util.Calendar; /** * Typical-use test binder for integration test resource configs. @@ -41,11 +42,14 @@ public StandardTestBinder(final AuditLogger auditLogger, final ServiceLocator in @Override protected void configure() { + EntityDictionary dictionary = new EntityDictionary(TestCheckMappings.MAPPINGS, injector::inject); + + bind(dictionary).to(EntityDictionary.class); + // Elide instance bindFactory(new Factory() { @Override public Elide provide() { - EntityDictionary dictionary = new EntityDictionary(TestCheckMappings.MAPPINGS, injector::inject); DefaultFilterDialect defaultFilterStrategy = new DefaultFilterDialect(dictionary); RSQLFilterDialect rsqlFilterStrategy = new RSQLFilterDialect(dictionary); @@ -59,6 +63,7 @@ public Elide provide() { .withJoinFilterDialect(multipleFilterStrategy) .withSubqueryFilterDialect(multipleFilterStrategy) .withEntityDictionary(dictionary) + .withISO8601Dates("yyyy-MM-dd'T'HH:mm'Z'", Calendar.getInstance().getTimeZone()) .build()); } diff --git a/elide-integration-tests/src/test/java/com/yahoo/elide/tests/GraphQLIT.java b/elide-integration-tests/src/test/java/com/yahoo/elide/tests/GraphQLIT.java index 1cb131c2d5..f91cd83297 100644 --- a/elide-integration-tests/src/test/java/com/yahoo/elide/tests/GraphQLIT.java +++ b/elide-integration-tests/src/test/java/com/yahoo/elide/tests/GraphQLIT.java @@ -37,7 +37,6 @@ import java.util.ArrayList; import java.util.Arrays; import java.util.Collection; -import java.util.Collections; import java.util.HashMap; import java.util.Map; import javax.ws.rs.core.MediaType; @@ -360,150 +359,6 @@ public void runUpdateAndFetchDifferentTransactionsBatch() throws IOException { ); } - @Test - public void runMultipleRequestsSameTransaction() throws IOException { - // This test demonstrates that multiple roots can be manipulated within a _single_ transaction - - String graphQLRequest = document( - selections( - field( - "book", - argument( - argument( - "ids", - Arrays.asList("1") - ) - ), - selections( - field("id"), - field("title"), - field( - "authors", - selections( - field("id"), - field("name") - ) - ) - ) - ), - field( - "author", - selections( - field("id"), - field("name") - ) - ) - ) - ).toQuery(); - - String expectedResponse = document( - selections( - field( - "book", - selections( - field("id", "1"), - field("title", "1984"), - field( - "authors", - selections( - field("id", "1"), - field("name", "George Orwell") - ) - ) - ) - ), - field( - "author", - selections( - field("id", "1"), - field("name", "George Orwell") - ) - ) - ) - ).toResponse(); - - runQueryWithExpectedResult(graphQLRequest, expectedResponse); - } - - @Test - public void runMultipleRequestsSameTransactionMutation() throws IOException { - // This test demonstrates that multiple roots can be manipulated within a _single_ transaction - // and results are consistent across a mutation. - Author author = new Author(); - author.setId(2L); - author.setName("Stephen King"); - - String graphQLRequest = document( - mutation( - selections( - field( - "book", - argument( - argument( - "ids", - Collections.singletonList("1") - ) - ), - selections( - field("id"), - field("title"), - field( - "authors", - arguments( - argument("op", "UPSERT"), - argument("data", author) - ), - selections( - field("id"), - field("name") - ) - ) - ) - ), - field( - "author", - selections( - field("id"), - field("name") - ) - ) - ) - ) - ).toQuery(); - - String expectedResponse = document( - selections( - field( - "book", - selections( - field("id", "1"), - field("title", "1984"), - field( - "authors", - selections( - field("id", "2"), - field("name", "Stephen King") - ) - ) - ) - ), - field( - "author", - selections( - field("id", "1"), - field("name", "George Orwell") - ), - selections( - field("id", "2"), - field("name", "Stephen King") - ) - ) - ) - ).toResponse(); - - runQueryWithExpectedResult(graphQLRequest, expectedResponse); - } - @Test public void runMultipleRequestsSameTransactionWithAliases() throws IOException { // This test demonstrates that multiple roots can be manipulated within a _single_ transaction diff --git a/elide-integration-tests/src/test/java/com/yahoo/elide/tests/ResourceIT.java b/elide-integration-tests/src/test/java/com/yahoo/elide/tests/ResourceIT.java index c55f7442ae..94b8a85bf7 100644 --- a/elide-integration-tests/src/test/java/com/yahoo/elide/tests/ResourceIT.java +++ b/elide-integration-tests/src/test/java/com/yahoo/elide/tests/ResourceIT.java @@ -48,6 +48,7 @@ import com.yahoo.elide.core.pagination.Pagination; import com.yahoo.elide.initialization.IntegrationTest; import com.yahoo.elide.jsonapi.models.JsonApiDocument; +import com.yahoo.elide.request.EntityProjection; import com.yahoo.elide.security.executors.BypassPermissionExecutor; import com.yahoo.elide.utils.JsonParser; @@ -75,7 +76,6 @@ import java.io.IOException; import java.util.HashMap; import java.util.HashSet; -import java.util.Optional; import java.util.Set; import java.util.stream.Stream; @@ -915,7 +915,7 @@ public void testGetIncludeBadRelation() { .accept(JSONAPI_CONTENT_TYPE) .get("/parent/1?include=children.BadRelation") .then() - .statusCode(HttpStatus.SC_NOT_FOUND); + .statusCode(HttpStatus.SC_BAD_REQUEST); } @Test @@ -2676,7 +2676,12 @@ public void testSpecialCharacterLikeQueryHQL(FilterPredicate filterPredicate, in when(scope.getDictionary()).thenReturn(dictionary); Pagination pagination = mock(Pagination.class); when(pagination.isGenerateTotals()).thenReturn(true); - tx.loadObjects(Book.class, Optional.of(filterPredicate), Optional.empty(), Optional.of(pagination), scope); + tx.loadObjects(EntityProjection.builder() + .type(Book.class) + + .filterExpression(filterPredicate) + .pagination(pagination) + .build(), scope); tx.commit(scope); tx.close(); verify(pagination).setPageTotals(noOfRecords); diff --git a/elide-integration-tests/src/test/resources/EncodedErrorResponsesIT/graphQLFetchError.json b/elide-integration-tests/src/test/resources/EncodedErrorResponsesIT/graphQLFetchError.json deleted file mode 100644 index 65e8020780..0000000000 --- a/elide-integration-tests/src/test/resources/EncodedErrorResponsesIT/graphQLFetchError.json +++ /dev/null @@ -1,10 +0,0 @@ -{ - "errors":[ - { - "message":"Invalid Syntax", - "locations":[ - {"line":1,"column":43} - ] - } - ] -} diff --git a/elide-integration-tests/src/test/resources/EncodedErrorResponsesIT/graphQLFetchErrorObjectEncoded.json b/elide-integration-tests/src/test/resources/EncodedErrorResponsesIT/graphQLFetchErrorObjectEncoded.json new file mode 100644 index 0000000000..3a66d8d63d --- /dev/null +++ b/elide-integration-tests/src/test/resources/EncodedErrorResponsesIT/graphQLFetchErrorObjectEncoded.json @@ -0,0 +1,7 @@ +{ + "errors":[ + { + "message": "InvalidEntityBodyException: Bad Request Body'Can't parse query: {invoice(sort: "<script>"){edges{node{id}}}'" + } + ] +} diff --git a/elide-integration-tests/src/test/resources/EncodedErrorResponsesIT/graphQLFetchErrorResponseEncoded.json b/elide-integration-tests/src/test/resources/EncodedErrorResponsesIT/graphQLFetchErrorResponseEncoded.json new file mode 100644 index 0000000000..85019c586a --- /dev/null +++ b/elide-integration-tests/src/test/resources/EncodedErrorResponsesIT/graphQLFetchErrorResponseEncoded.json @@ -0,0 +1,5 @@ +{ + "errors":[ + "InvalidEntityBodyException: Bad Request Body'Can't parse query: {invoice(sort: "<script>"){edges{node{id}}}'" + ] +} diff --git a/elide-spring/elide-spring-boot-autoconfigure/pom.xml b/elide-spring/elide-spring-boot-autoconfigure/pom.xml index 59859680d6..6d1c97613c 100644 --- a/elide-spring/elide-spring-boot-autoconfigure/pom.xml +++ b/elide-spring/elide-spring-boot-autoconfigure/pom.xml @@ -2,7 +2,7 @@ 4.0.0 com.yahoo.elide elide-spring-boot-autoconfigure - 4.6.2-SNAPSHOT + 5.0.0-pr6-SNAPSHOT jar Elide Spring Boot Autoconfigure Elide Spring Boot Autoconfigure @@ -10,7 +10,7 @@ com.yahoo.elide elide-spring-parent-pom - 4.6.2-SNAPSHOT + 5.0.0-pr6-SNAPSHOT @@ -54,35 +54,42 @@ com.yahoo.elide elide-core - 4.6.2-SNAPSHOT + 5.0.0-pr6-SNAPSHOT true com.yahoo.elide elide-graphql - 4.6.2-SNAPSHOT + 5.0.0-pr6-SNAPSHOT true com.yahoo.elide elide-annotations - 4.6.2-SNAPSHOT + 5.0.0-pr6-SNAPSHOT + true + + + + com.yahoo.elide + elide-datastore-aggregation + 5.0.0-pr6-SNAPSHOT true com.yahoo.elide elide-datastore-jpa - 4.6.2-SNAPSHOT + 5.0.0-pr6-SNAPSHOT true com.yahoo.elide elide-swagger - 4.6.2-SNAPSHOT + 5.0.0-pr6-SNAPSHOT true @@ -151,7 +158,7 @@ com.yahoo.elide elide-test-helpers - 4.6.2-SNAPSHOT + 5.0.0-pr6-SNAPSHOT test diff --git a/elide-spring/elide-spring-boot-autoconfigure/src/main/java/com/yahoo/elide/spring/config/ElideAutoConfiguration.java b/elide-spring/elide-spring-boot-autoconfigure/src/main/java/com/yahoo/elide/spring/config/ElideAutoConfiguration.java index 55d9b38fbb..29a10bf969 100644 --- a/elide-spring/elide-spring-boot-autoconfigure/src/main/java/com/yahoo/elide/spring/config/ElideAutoConfiguration.java +++ b/elide-spring/elide-spring-boot-autoconfigure/src/main/java/com/yahoo/elide/spring/config/ElideAutoConfiguration.java @@ -13,8 +13,13 @@ import com.yahoo.elide.core.DataStore; import com.yahoo.elide.core.EntityDictionary; import com.yahoo.elide.core.filter.dialect.RSQLFilterDialect; +import com.yahoo.elide.datastores.aggregation.AggregationDataStore; +import com.yahoo.elide.datastores.aggregation.QueryEngineFactory; +import com.yahoo.elide.datastores.aggregation.metadata.MetaDataStore; +import com.yahoo.elide.datastores.aggregation.queryengines.sql.SQLQueryEngineFactory; import com.yahoo.elide.datastores.jpa.JpaDataStore; import com.yahoo.elide.datastores.jpa.transaction.NonJtaTransaction; +import com.yahoo.elide.datastores.multiplex.MultiplexManager; import org.springframework.beans.factory.config.AutowireCapableBeanFactory; import org.springframework.boot.autoconfigure.condition.ConditionalOnMissingBean; @@ -96,11 +101,19 @@ public T instantiate(Class cls) { */ @Bean @ConditionalOnMissingBean - public DataStore buildDataStore(EntityManagerFactory entityManagerFactory) throws ClassNotFoundException { + public DataStore buildDataStore(EntityManagerFactory entityManagerFactory, + QueryEngineFactory queryEngineFactory, + ElideConfigProperties settings) throws ClassNotFoundException { + MetaDataStore metaDataStore = new MetaDataStore(); - return new JpaDataStore( + AggregationDataStore aggregationDataStore = new AggregationDataStore(queryEngineFactory, metaDataStore); + + JpaDataStore jpaDataStore = new JpaDataStore( () -> { return entityManagerFactory.createEntityManager(); }, (em -> { return new NonJtaTransaction(em); })); + + // meta data store needs to be put at first to populate meta data models + return new MultiplexManager(jpaDataStore, metaDataStore, aggregationDataStore); } /** @@ -122,4 +135,15 @@ public Swagger buildSwagger(EntityDictionary dictionary, ElideConfigProperties s return swagger; } + + /** + * Configure the QueryEngineFactory that the Aggregation Data Store uses. + * @param entityManagerFactory Needed by the SQLQueryEngine + * @return a SQLQueryEngineFactory + */ + @Bean + @ConditionalOnMissingBean + public QueryEngineFactory buildQueryEngineFactory(EntityManagerFactory entityManagerFactory) { + return new SQLQueryEngineFactory(entityManagerFactory); + } } diff --git a/elide-spring/elide-spring-boot-autoconfigure/src/test/java/com/yahoo/elide/spring/models/aggregation/Stats.java b/elide-spring/elide-spring-boot-autoconfigure/src/test/java/com/yahoo/elide/spring/models/aggregation/Stats.java new file mode 100644 index 0000000000..df6feca4b6 --- /dev/null +++ b/elide-spring/elide-spring-boot-autoconfigure/src/test/java/com/yahoo/elide/spring/models/aggregation/Stats.java @@ -0,0 +1,42 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.spring.models.aggregation; + +import com.yahoo.elide.annotation.Include; +import com.yahoo.elide.datastores.aggregation.annotation.Cardinality; +import com.yahoo.elide.datastores.aggregation.annotation.CardinalitySize; +import com.yahoo.elide.datastores.aggregation.annotation.MetricAggregation; +import com.yahoo.elide.datastores.aggregation.queryengines.sql.annotation.FromTable; +import com.yahoo.elide.datastores.aggregation.queryengines.sql.metric.functions.SqlSum; +import lombok.EqualsAndHashCode; +import lombok.ToString; + +import javax.persistence.Id; + +@Include(rootLevel = true) +@Cardinality(size = CardinalitySize.LARGE) +@EqualsAndHashCode +@ToString +@FromTable(name = "stats") +public class Stats { + + /** + * PK. + */ + @Id + private String id; + + /** + * A metric. + */ + @MetricAggregation(function = SqlSum.class) + private long measure; + + /** + * A degenerate dimension. + */ + private String dimension; +} diff --git a/elide-spring/elide-spring-boot-autoconfigure/src/test/java/com/yahoo/elide/spring/models/ArtifactGroup.java b/elide-spring/elide-spring-boot-autoconfigure/src/test/java/com/yahoo/elide/spring/models/jpa/ArtifactGroup.java similarity index 95% rename from elide-spring/elide-spring-boot-autoconfigure/src/test/java/com/yahoo/elide/spring/models/ArtifactGroup.java rename to elide-spring/elide-spring-boot-autoconfigure/src/test/java/com/yahoo/elide/spring/models/jpa/ArtifactGroup.java index 0277a2eebf..30b846bd50 100644 --- a/elide-spring/elide-spring-boot-autoconfigure/src/test/java/com/yahoo/elide/spring/models/ArtifactGroup.java +++ b/elide-spring/elide-spring-boot-autoconfigure/src/test/java/com/yahoo/elide/spring/models/jpa/ArtifactGroup.java @@ -3,7 +3,7 @@ * Licensed under the Apache License, Version 2.0 * See LICENSE file in project root for terms. */ -package com.yahoo.elide.spring.models; +package com.yahoo.elide.spring.models.jpa; import com.yahoo.elide.annotation.CreatePermission; import com.yahoo.elide.annotation.Include; diff --git a/elide-spring/elide-spring-boot-autoconfigure/src/test/java/com/yahoo/elide/spring/models/ArtifactProduct.java b/elide-spring/elide-spring-boot-autoconfigure/src/test/java/com/yahoo/elide/spring/models/jpa/ArtifactProduct.java similarity index 94% rename from elide-spring/elide-spring-boot-autoconfigure/src/test/java/com/yahoo/elide/spring/models/ArtifactProduct.java rename to elide-spring/elide-spring-boot-autoconfigure/src/test/java/com/yahoo/elide/spring/models/jpa/ArtifactProduct.java index 7c75d6d5d0..73158a14ec 100644 --- a/elide-spring/elide-spring-boot-autoconfigure/src/test/java/com/yahoo/elide/spring/models/ArtifactProduct.java +++ b/elide-spring/elide-spring-boot-autoconfigure/src/test/java/com/yahoo/elide/spring/models/jpa/ArtifactProduct.java @@ -3,7 +3,7 @@ * Licensed under the Apache License, Version 2.0 * See LICENSE file in project root for terms. */ -package com.yahoo.elide.spring.models; +package com.yahoo.elide.spring.models.jpa; import com.yahoo.elide.annotation.Include; diff --git a/elide-spring/elide-spring-boot-autoconfigure/src/test/java/com/yahoo/elide/spring/models/ArtifactVersion.java b/elide-spring/elide-spring-boot-autoconfigure/src/test/java/com/yahoo/elide/spring/models/jpa/ArtifactVersion.java similarity index 92% rename from elide-spring/elide-spring-boot-autoconfigure/src/test/java/com/yahoo/elide/spring/models/ArtifactVersion.java rename to elide-spring/elide-spring-boot-autoconfigure/src/test/java/com/yahoo/elide/spring/models/jpa/ArtifactVersion.java index 8ffa0cd6e6..b31c64ec2e 100644 --- a/elide-spring/elide-spring-boot-autoconfigure/src/test/java/com/yahoo/elide/spring/models/ArtifactVersion.java +++ b/elide-spring/elide-spring-boot-autoconfigure/src/test/java/com/yahoo/elide/spring/models/jpa/ArtifactVersion.java @@ -3,7 +3,7 @@ * Licensed under the Apache License, Version 2.0 * See LICENSE file in project root for terms. */ -package com.yahoo.elide.spring.models; +package com.yahoo.elide.spring.models.jpa; import com.yahoo.elide.annotation.Include; diff --git a/elide-spring/elide-spring-boot-autoconfigure/src/test/java/com/yahoo/elide/spring/tests/AggregationStoreTest.java b/elide-spring/elide-spring-boot-autoconfigure/src/test/java/com/yahoo/elide/spring/tests/AggregationStoreTest.java new file mode 100644 index 0000000000..ec6395e513 --- /dev/null +++ b/elide-spring/elide-spring-boot-autoconfigure/src/test/java/com/yahoo/elide/spring/tests/AggregationStoreTest.java @@ -0,0 +1,54 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.spring.tests; + +import static com.jayway.restassured.RestAssured.when; +import static com.yahoo.elide.contrib.testhelpers.jsonapi.JsonApiDSL.attr; +import static com.yahoo.elide.contrib.testhelpers.jsonapi.JsonApiDSL.attributes; +import static com.yahoo.elide.contrib.testhelpers.jsonapi.JsonApiDSL.data; +import static com.yahoo.elide.contrib.testhelpers.jsonapi.JsonApiDSL.id; +import static com.yahoo.elide.contrib.testhelpers.jsonapi.JsonApiDSL.resource; +import static com.yahoo.elide.contrib.testhelpers.jsonapi.JsonApiDSL.type; +import static org.hamcrest.Matchers.equalTo; + +import com.yahoo.elide.core.HttpStatus; + +import org.junit.jupiter.api.Test; +import org.springframework.test.context.jdbc.Sql; + +/** + * Example functional tests for Aggregation Store. + */ +public class AggregationStoreTest extends IntegrationTest { + /** + * This test demonstrates an example test using the aggregation store. + */ + @Test + @Sql(statements = { + "DROP TABLE Stats IF EXISTS;", + "CREATE TABLE Stats(id int, measure int, dimension VARCHAR(255));", + "INSERT INTO Stats (id, measure, dimension) VALUES\n" + + "\t\t(1,100,'Foo')," + + "\t\t(2,150,'Bar');" + }) + public void jsonApiGetTest() { + when() + .get("/json/stats?fields[stats]=measure") + .then() + .body(equalTo( + data( + resource( + type("stats"), + id("0"), + attributes( + attr("measure", 250) + ) + ) + ).toJSON()) + ) + .statusCode(HttpStatus.SC_OK); + } +} diff --git a/elide-spring/elide-spring-boot-autoconfigure/src/test/java/com/yahoo/elide/spring/tests/ControllerTest.java b/elide-spring/elide-spring-boot-autoconfigure/src/test/java/com/yahoo/elide/spring/tests/ControllerTest.java index 34f52833b4..bd0bdb813c 100644 --- a/elide-spring/elide-spring-boot-autoconfigure/src/test/java/com/yahoo/elide/spring/tests/ControllerTest.java +++ b/elide-spring/elide-spring-boot-autoconfigure/src/test/java/com/yahoo/elide/spring/tests/ControllerTest.java @@ -32,7 +32,8 @@ import com.yahoo.elide.contrib.testhelpers.graphql.GraphQLDSL; import com.yahoo.elide.core.HttpStatus; import com.yahoo.elide.spring.controllers.JsonApiController; -import com.yahoo.elide.spring.models.ArtifactGroup; +import com.yahoo.elide.spring.models.jpa.ArtifactGroup; + import org.junit.jupiter.api.Test; import org.springframework.test.context.jdbc.Sql; import org.springframework.test.context.jdbc.SqlMergeMode; diff --git a/elide-spring/elide-spring-boot-autoconfigure/src/test/resources/application.yaml b/elide-spring/elide-spring-boot-autoconfigure/src/test/resources/application.yaml index a108b0f6ea..2af8bdec42 100644 --- a/elide-spring/elide-spring-boot-autoconfigure/src/test/resources/application.yaml +++ b/elide-spring/elide-spring-boot-autoconfigure/src/test/resources/application.yaml @@ -2,6 +2,7 @@ server: port: 4001 elide: + modelPackage: 'com.yahoo.elide.spring.models' json-api: path: /json enabled: true @@ -25,7 +26,6 @@ spring: naming: physical-strategy: 'org.hibernate.boot.model.naming.PhysicalNamingStrategyStandardImpl' ddl-auto: 'create' - datasource: url: 'jdbc:h2:mem:db1;DB_CLOSE_DELAY=-1' username: 'sa' diff --git a/elide-spring/elide-spring-boot-starter/pom.xml b/elide-spring/elide-spring-boot-starter/pom.xml index dc42d68b3c..e6ee748c5e 100644 --- a/elide-spring/elide-spring-boot-starter/pom.xml +++ b/elide-spring/elide-spring-boot-starter/pom.xml @@ -2,7 +2,7 @@ 4.0.0 com.yahoo.elide elide-spring-boot-starter - 4.6.2-SNAPSHOT + 5.0.0-pr6-SNAPSHOT jar Elide Spring Boot Starter Elide Spring Boot Starter @@ -10,7 +10,7 @@ com.yahoo.elide elide-spring-parent-pom - 4.6.2-SNAPSHOT + 5.0.0-pr6-SNAPSHOT @@ -50,31 +50,37 @@ com.yahoo.elide elide-core - 4.6.2-SNAPSHOT + 5.0.0-pr6-SNAPSHOT com.yahoo.elide elide-graphql - 4.6.2-SNAPSHOT + 5.0.0-pr6-SNAPSHOT com.yahoo.elide elide-annotations - 4.6.2-SNAPSHOT + 5.0.0-pr6-SNAPSHOT com.yahoo.elide elide-datastore-jpa - 4.6.2-SNAPSHOT + 5.0.0-pr6-SNAPSHOT + + + + com.yahoo.elide + elide-datastore-aggregation + 5.0.0-pr6-SNAPSHOT com.yahoo.elide elide-swagger - 4.6.2-SNAPSHOT + 5.0.0-pr6-SNAPSHOT @@ -97,7 +103,7 @@ com.yahoo.elide elide-spring-boot-autoconfigure - 4.6.2-SNAPSHOT + 5.0.0-pr6-SNAPSHOT @@ -114,6 +120,24 @@ + + + org.eclipse.jetty + jetty-servlets + ${version.jetty} + + + org.eclipse.jetty.websocket + websocket-server + ${version.jetty} + + + org.eclipse.jetty.websocket + javax-websocket-server-impl + ${version.jetty} + + + org.springframework.boot diff --git a/elide-spring/pom.xml b/elide-spring/pom.xml index 2b00dc9074..359b3e0c4d 100644 --- a/elide-spring/pom.xml +++ b/elide-spring/pom.xml @@ -14,7 +14,7 @@ elide-parent-pom com.yahoo.elide - 4.6.2-SNAPSHOT + 5.0.0-pr6-SNAPSHOT diff --git a/elide-standalone/pom.xml b/elide-standalone/pom.xml index c6e0b0314d..d06be32521 100644 --- a/elide-standalone/pom.xml +++ b/elide-standalone/pom.xml @@ -8,7 +8,7 @@ 4.0.0 com.yahoo.elide elide-standalone - 4.6.2-SNAPSHOT + 5.0.0-pr6-SNAPSHOT jar Elide Standalone Elide Standalone Application @@ -16,7 +16,7 @@ com.yahoo.elide elide-parent-pom - 4.6.2-SNAPSHOT + 5.0.0-pr6-SNAPSHOT @@ -61,17 +61,17 @@ com.yahoo.elide elide-datastore-jpa - 4.6.2-SNAPSHOT + 5.0.0-pr6-SNAPSHOT com.yahoo.elide elide-graphql - 4.6.2-SNAPSHOT + 5.0.0-pr6-SNAPSHOT com.yahoo.elide elide-swagger - 4.6.2-SNAPSHOT + 5.0.0-pr6-SNAPSHOT @@ -201,7 +201,7 @@ com.yahoo.elide elide-test-helpers - 4.6.2-SNAPSHOT + 5.0.0-pr6-SNAPSHOT test diff --git a/elide-standalone/src/main/java/com/yahoo/elide/standalone/ElideStandalone.java b/elide-standalone/src/main/java/com/yahoo/elide/standalone/ElideStandalone.java index f3ba3533bf..78e400c899 100644 --- a/elide-standalone/src/main/java/com/yahoo/elide/standalone/ElideStandalone.java +++ b/elide-standalone/src/main/java/com/yahoo/elide/standalone/ElideStandalone.java @@ -111,7 +111,7 @@ public void start(boolean block) throws Exception { if (elideStandaloneSettings.enableGraphQL()) { ServletHolder jerseyServlet = context.addServlet(ServletContainer.class, - elideStandaloneSettings.getGraphQLApiPathSepc()); + elideStandaloneSettings.getGraphQLApiPathSpec()); jerseyServlet.setInitOrder(0); jerseyServlet.setInitParameter("jersey.config.server.provider.packages", "com.yahoo.elide.graphql"); jerseyServlet.setInitParameter("javax.ws.rs.Application", ElideResourceConfig.class.getCanonicalName()); @@ -135,7 +135,7 @@ public void start(boolean block) throws Exception { if (!elideStandaloneSettings.enableSwagger().isEmpty()) { ServletHolder jerseyServlet = context.addServlet(ServletContainer.class, - elideStandaloneSettings.getSwaggerPathSepc()); + elideStandaloneSettings.getSwaggerPathSpec()); jerseyServlet.setInitOrder(0); jerseyServlet.setInitParameter("jersey.config.server.provider.packages", "com.yahoo.elide.contrib.swagger.resources"); diff --git a/elide-standalone/src/main/java/com/yahoo/elide/standalone/config/ElideStandaloneSettings.java b/elide-standalone/src/main/java/com/yahoo/elide/standalone/config/ElideStandaloneSettings.java index e96edc3ad8..66d05be847 100644 --- a/elide-standalone/src/main/java/com/yahoo/elide/standalone/config/ElideStandaloneSettings.java +++ b/elide-standalone/src/main/java/com/yahoo/elide/standalone/config/ElideStandaloneSettings.java @@ -91,7 +91,7 @@ public T instantiate(Class cls) { .withSubqueryFilterDialect(new RSQLFilterDialect(dictionary)) .withAuditLogger(getAuditLogger()); - if (enableIS06081Dates()) { + if (enableISO8601Dates()) { builder = builder.withISO8601Dates("yyyy-MM-dd'T'HH:mm'Z'", TimeZone.getTimeZone("UTC")); } @@ -148,8 +148,8 @@ default String getJsonApiPathSpec() { * * @return Default: /graphql/api/v1 */ - default String getGraphQLApiPathSepc() { - return "/graphql/api/v1"; + default String getGraphQLApiPathSpec() { + return "/graphql/api/v1/*"; } @@ -158,7 +158,7 @@ default String getGraphQLApiPathSepc() { * * @return Default: /swagger/* */ - default String getSwaggerPathSepc() { + default String getSwaggerPathSpec() { return "/swagger/*"; } @@ -184,7 +184,7 @@ default boolean enableGraphQL() { * Whether Dates should be ISO8601 strings (true) or epochs (false). * @return */ - default boolean enableIS06081Dates() { + default boolean enableISO8601Dates() { return true; } diff --git a/pom.xml b/pom.xml index ebec64b99e..4378d1eaf9 100644 --- a/pom.xml +++ b/pom.xml @@ -8,7 +8,7 @@ 4.0.0 com.yahoo.elide elide-parent-pom - 4.6.2-SNAPSHOT + 5.0.0-pr6-SNAPSHOT pom Elide: Parent Pom Parent pom for Elide project @@ -103,12 +103,12 @@ com.yahoo.elide elide-annotations - 4.6.2-SNAPSHOT + 5.0.0-pr6-SNAPSHOT com.yahoo.elide elide-example-models - 4.6.2-SNAPSHOT + 5.0.0-pr6-SNAPSHOT org.projectlombok