Skip to content

Commit

Permalink
[2.x][Manual Backport ] Back port main to 2.x (#7477) (#7487)
Browse files Browse the repository at this point in the history
* Bump OUI to 1.8.0 (#7363)


(cherry picked from commit c1bae75)

* [Auto Suggest] OpenSearch SQL autosuggest with ANTLR (#7336)

* cherry pick



* SQL keyword suggestion



* column and values suggestions



* adjust values suggestions and ignored token



* suggestion type casting



* ignore functions



* case insensitive



* change to use grammar for insensitive case



* fix multi-line issue



* code cleanup



* move ppl out of scope



* rename folder and add ignoring rules



* resolve type issue



* fix ppl suggestion provider issue



* remove function suggestion



* some code clean up and adding tests



* cursor tests



* remove testing setup



* add changelog



* update yarn file



* add missing testing library



* MDS integration



* minor interface change and disable word based suggestion



* update ID_LITERAL and recompile grammar



* column suggest for agg function



* revert version



* opensearch sql syntax highlighting



* add utility tests



* symbol table test



* error listener test



* parse test



* add suggestion provider to single line query editor



* remove one test



* add connection service type



---------


(cherry picked from commit 9348bd4)

* Move TopNavLinks to new Nav Bar Discover (#7326)

This PR aims to introduce a new Nav bar in Discover that would contain the Date Picker as well as the Navigation links that are currently part of headers. The Navigation Links have been replaced with the corresponding Icons to provide a modern UX. These setting are currently controlled by query:enhancements:enabled Advanced Setting Flag and would be visible when the above flag is turned on.


(cherry picked from commit e46e54a)

* [navigation-next]fix: breadcrumb issue found in data source management and other similar cases. (#7401)

* feat: add scoped breadcrumbs



* Changeset file for PR #7401 created/updated

* feat: move data source management register logic up a little bit



* feat: add unit test



* feat: revert the home related change



* fix: overview error in all use case when workspace is enabled



* fix: update snapshot



* Changeset file for PR #7401 created/updated

* fix: hide nav groups that should be displayed



* feat: hide expand icon in left navigation



* feat: update



---------




(cherry picked from commit 3f78c94)

* [Bug][Data Source] Move data source manageable feature flag to DSM plugin (#7440)

* Move data source manageable feature flag to DSM plugin



* Changeset file for PR #7440 created/updated

---------



(cherry picked from commit dfcd2e1)

* [Discover-next] data set picker (#7426)

* update using query manager



* aggs not working



* almost there



* stablish



* thanks ashwin



* update ref name



* fix timefields



* clean up some console logs



* go safer route of setting language



* its working again



* fix names



* restore code editor and indices



* sql df



* fix external datasources again



---------


(cherry picked from commit 6a079d3)

* Fix add filter for numeric scripted field (#7022)

* Fix add filter for scripted field



* add unit test



* change from nested ternary to if else



* add more unit tests



* change unit tests



* fix typo



---------


(cherry picked from commit 7c1f8fc)

* [Bug][Workspace] Add permission validation at workspace detail page (#7435)

* [Bug][Workspace] Add permission validate at workspace detail page



* Changeset file for PR #7435 created/updated

---------



(cherry picked from commit 45d34b5)

* Revert "[Discover-next] data set picker (#7426)" (#7479)

This reverts commit 6a079d3.

---------








(cherry picked from commit 0515fb2)

Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Miki <miki@amazon.com>
Co-authored-by: Eric Wei <menwe@amazon.com>
Co-authored-by: SuZhou-Joe <suzhou@amazon.com>
Co-authored-by: yuboluo <yubonluo@amazon.com>
Co-authored-by: Kawika Avilla <kavilla414@gmail.com>
Co-authored-by: Qingyang(Abby) Hu <abigailhu2000@gmail.com>
Co-authored-by: Sean Li <lnse@amazon.com>
  • Loading branch information
9 people authored Jul 25, 2024
1 parent 95b2ef6 commit 975d670
Show file tree
Hide file tree
Showing 76 changed files with 44,032 additions and 115 deletions.
4 changes: 4 additions & 0 deletions .eslintignore
Original file line number Diff line number Diff line change
Expand Up @@ -30,3 +30,7 @@ target
/packages/osd-test/src/functional_test_runner/lib/config/__tests__/fixtures/
/packages/osd-ui-framework/dist
/packages/osd-ui-shared-deps/flot_charts

# antlr overrides
/src/plugins/data/public/antlr/opensearch_sql/.generated/*
/src/plugins/data/public/antlr/opensearch_sql/grammar/**/*
9 changes: 9 additions & 0 deletions .eslintrc.js
Original file line number Diff line number Diff line change
Expand Up @@ -751,5 +751,14 @@ module.exports = {
'no-undef': 'off',
},
},
{
files: [
'src/plugins/data/public/antlr/opensearch_sql/.generated/*',
'src/plugins/data/public/antlr/opensearch_sql/grammar/**/*',
],
rules: {
'filenames/match-regex': 'off',
},
},
],
};
2 changes: 2 additions & 0 deletions changelogs/fragments/7336.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
feat:
- [Auto Suggest] OpenSearch SQL autosuggest with ANTLR ([#7336](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/7336))
2 changes: 2 additions & 0 deletions changelogs/fragments/7363.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
feat:
- Bump OUI to 1.8.0 ([#7363](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/7363))
2 changes: 1 addition & 1 deletion config/opensearch_dashboards.yml
Original file line number Diff line number Diff line change
Expand Up @@ -362,4 +362,4 @@
# Set the backend roles in groups or users, whoever has the backend roles or exactly match the user ids defined in this config will be regard as dashboard admin.
# Dashboard admin will have the access to all the workspaces(workspace.enabled: true) and objects inside OpenSearch Dashboards.
# opensearchDashboards.dashboardAdmin.groups: ["dashboard_admin"]
# opensearchDashboards.dashboardAdmin.users: ["dashboard_admin"]
# opensearchDashboards.dashboardAdmin.users: ["dashboard_admin"]
19 changes: 15 additions & 4 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,12 @@
"osd:bootstrap": "scripts/use_node scripts/build_ts_refs && scripts/use_node scripts/register_git_hook",
"spec_to_console": "scripts/use_node scripts/spec_to_console",
"pkg-version": "scripts/use_node -e \"console.log(require('./package.json').version)\"",
"release_note:generate": "scripts/use_node scripts/generate_release_note"
"release_note:generate": "scripts/use_node scripts/generate_release_note",
"cypress:run-without-security": "env TZ=America/Los_Angeles NO_COLOR=1 cypress run --headless --env SECURITY_ENABLED=false",
"cypress:run-with-security": "env TZ=America/Los_Angeles NO_COLOR=1 cypress run --headless --env SECURITY_ENABLED=true,openSearchUrl=https://localhost:9200,WAIT_FOR_LOADER_BUFFER_MS=500",
"osd:ciGroup10": "echo \"dashboard_sanity_test_spec.js\"",
"osd:ciGroup11": "echo \"apps/vis_builder/*.js\"",
"generate:opensearchsqlantlr": "./node_modules/antlr4ng-cli/index.js -Dlanguage=TypeScript -o ./src/plugins/data/public/antlr/opensearch_sql/.generated -visitor -no-listener -Xexact-output-dir ./src/plugins/data/public/antlr/opensearch_sql/grammar/OpenSearchSQLLexer.g4 ./src/plugins/data/public/antlr/opensearch_sql/grammar/OpenSearchSQLParser.g4"
},
"repository": {
"type": "git",
Expand Down Expand Up @@ -109,7 +114,7 @@
"**/qs": "^6.11.0",
"**/semver": "^7.5.3",
"**/set-value": "^4.1.0",
"**/tar":"^6.2.1",
"**/tar": "^6.2.1",
"**/topo/hoek": "npm:@amoo-miki/hoek@6.1.3",
"**/trim": "^0.0.3",
"**/typescript": "4.0.2",
Expand Down Expand Up @@ -140,7 +145,7 @@
"dependencies": {
"@aws-crypto/client-node": "^3.1.1",
"@elastic/datemath": "5.0.3",
"@elastic/eui": "npm:@opensearch-project/oui@1.7.0",
"@elastic/eui": "npm:@opensearch-project/oui@1.8.0",
"@elastic/good": "^9.0.1-kibana3",
"@elastic/numeral": "npm:@amoo-miki/numeral@2.6.0",
"@elastic/request-crypto": "2.0.0",
Expand Down Expand Up @@ -177,6 +182,8 @@
"JSONStream": "1.3.5",
"abortcontroller-polyfill": "^1.4.0",
"ajv": "^8.11.0",
"antlr4-c3": "^3.4.1",
"antlr4ng": "^3.0.4",
"bluebird": "3.5.5",
"chalk": "^4.1.0",
"chokidar": "^3.4.2",
Expand Down Expand Up @@ -242,11 +249,13 @@
"type-detect": "^4.0.8",
"uuid": "3.3.2",
"whatwg-fetch": "^3.0.0",
"yauzl": "^2.10.0"
"yauzl": "^2.10.0",
"@opensearch-dashboards-test/opensearch-dashboards-test-library": "https://github.com/opensearch-project/opensearch-dashboards-test-library/archive/refs/tags/1.0.6.tar.gz"
},
"devDependencies": {
"@babel/core": "^7.22.9",
"@babel/parser": "^7.22.9",
"@babel/plugin-transform-class-static-block": "^7.24.4",
"@babel/register": "^7.22.9",
"@babel/types": "^7.22.9",
"@elastic/apm-rum": "^5.6.1",
Expand Down Expand Up @@ -361,6 +370,7 @@
"@types/zen-observable": "^0.8.0",
"@typescript-eslint/eslint-plugin": "^3.10.0",
"@typescript-eslint/parser": "^3.10.0",
"antlr4ng-cli": "^2.0.0",
"archiver": "^5.3.0",
"axe-core": "^4.0.2",
"babel-eslint": "^10.0.3",
Expand Down Expand Up @@ -461,6 +471,7 @@
"regenerate": "^1.4.0",
"reselect": "^4.0.0",
"resize-observer-polyfill": "^1.5.1",
"rimraf": "^5.0.7",
"selenium-webdriver": "^4.0.0-alpha.7",
"simple-git": "^3.16.0",
"sinon": "^7.4.2",
Expand Down
6 changes: 6 additions & 0 deletions packages/osd-babel-preset/webpack_preset.js
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,12 @@ module.exports = () => {
fileName: false,
},
],
[
require.resolve('@babel/plugin-transform-class-static-block'),
{
fileName: false,
},
],
],
env: {
production: {
Expand Down
3 changes: 3 additions & 0 deletions packages/osd-monaco/src/xjson/lexer_rules/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@ import * as monaco from 'monaco-editor/esm/vs/editor/editor.api';
import * as xJson from './xjson';
import * as opensearchql from './opensearchql';
import * as painless from './painless';
import * as opensearchsql from './opensearchsql';

export const registerLexerRules = (m: typeof monaco) => {
m.languages.register({ id: xJson.ID });
Expand All @@ -41,4 +42,6 @@ export const registerLexerRules = (m: typeof monaco) => {
m.languages.setMonarchTokensProvider(painless.ID, painless.lexerRules);
m.languages.register({ id: opensearchql.ID });
m.languages.setMonarchTokensProvider(opensearchql.ID, opensearchql.lexerRules);
m.languages.register({ id: opensearchsql.ID });
m.languages.setMonarchTokensProvider(opensearchsql.ID, opensearchsql.lexerRules);
};
157 changes: 157 additions & 0 deletions packages/osd-monaco/src/xjson/lexer_rules/opensearchsql.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,157 @@
/*
* Copyright OpenSearch Contributors
* SPDX-License-Identifier: Apache-2.0
*/

import { monaco } from '../../monaco';

export const ID = 'SQL';

const keywords = [
'ALL',
'AND',
'AS',
'ASC',
'BOOLEAN',
'BETWEEN',
'BY',
'CASE',
'CAST',
'CROSS',
'COLUMNS',
'DATETIME',
'DELETE',
'DESC',
'DESCRIBE',
'DISTINCT',
'DOUBLE',
'ELSE',
'EXISTS',
'FALSE',
'FLOAT',
'FIRST',
'FROM',
'GROUP',
'HAVING',
'IN',
'INNER',
'INT',
'INTEGER',
'IS',
'JOIN',
'LAST',
'LEFT',
'LIKE',
'LIMIT',
'LONG',
'MATCH',
'NATURAL',
'NOT',
'NULL',
'NULLS',
'ON',
'OR',
'ORDER',
'OUTER',
'OVER',
'PARTITION',
'REGEXP',
'RIGHT',
'SELECT',
'SHOW',
'STRING',
'THEN',
'TRUE',
'UNION',
'USING',
'WHEN',
'WHERE',
'EXCEPT',
];

const functions = [
'AVG',
'COUNT',
'MAX',
'MIN',
'SUM',
'VAR_POP',
'VAR_SAMP',
'VARIANCE',
'STD',
'STDDEV',
'STDDEV_POP',
'STDDEV_SAMP',
'SUBSTRING',
'TRIM',
];

const operators = [
'=',
'>',
'<',
'!',
'~',
'\\|',
'&',
'\\^',
'\\*',
'/',
'%',
'\\+',
'-',
'DIV',
'MOD',
];

const brackets = [
{ open: '(', close: ')', token: 'delimiter.parenthesis' },
{ open: '[', close: ']', token: 'delimiter.square' },
];

export const lexerRules = {
defaultToken: 'invalid',
ignoreCase: true,
tokenPostfix: '',
keywords,
functions,
operators,
brackets,
tokenizer: {
root: [
[
/[a-zA-Z_$][a-zA-Z0-9_$]*/,
{
cases: {
'@keywords': 'keyword',
'@functions': 'function',
'@default': 'identifier',
},
},
],
{ include: '@whitespace' },
[/[()]/, '@brackets'],
[new RegExp(operators.join('|')), 'operator'],
[/[0-9]+(\.[0-9]+)?/, 'number'],
[/'([^'\\]|\\.)*$/, 'string.invalid'], // non-terminated string
[/'/, 'string', '@string'],
[/"/, 'string', '@string'],
],
whitespace: [
[/[ \t\r\n]+/, 'white'],
[/\/\*/, 'comment', '@comment'],
[/--.*$/, 'comment'],
],
string: [
[/[^'\\]+/, 'string'],
[/\\./, 'string.escape'],
[/'/, 'string', '@pop'],
[/"/, 'string', '@pop'],
],
comment: [
[/[^/*]+/, 'comment'],
[/\*\//, 'comment', '@pop'],
[/[\/*]/, 'comment'],
],
},
} as monaco.languages.IMonarchLanguage;
10 changes: 10 additions & 0 deletions packages/osd-optimizer/src/worker/webpack.config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -256,6 +256,16 @@ export function getWebpackConfig(bundle: Bundle, bundleRefs: BundleRefs, worker:
loader: 'raw-loader',
},
},
{
test: /\.cjs$/,
include: /node_modules/,
use: {
loader: 'babel-loader',
options: {
presets: [BABEL_PRESET_PATH],
},
},
},
],
},

Expand Down
2 changes: 1 addition & 1 deletion packages/osd-ui-framework/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@
"enzyme-adapter-react-16": "^1.9.1"
},
"devDependencies": {
"@elastic/eui": "npm:@opensearch-project/oui@1.7.0",
"@elastic/eui": "npm:@opensearch-project/oui@1.8.0",
"@osd/babel-preset": "1.0.0",
"@osd/optimizer": "1.0.0",
"comment-stripper": "^0.0.4",
Expand Down
2 changes: 1 addition & 1 deletion packages/osd-ui-shared-deps/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
},
"dependencies": {
"@elastic/charts": "31.1.0",
"@elastic/eui": "npm:@opensearch-project/oui@1.7.0",
"@elastic/eui": "npm:@opensearch-project/oui@1.8.0",
"@elastic/numeral": "npm:@amoo-miki/numeral@2.6.0",
"@opensearch/datemath": "5.0.3",
"@osd/i18n": "1.0.0",
Expand Down
2 changes: 2 additions & 0 deletions src/core/public/application/types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -539,6 +539,8 @@ export interface AppMountParameters<HistoryLocationState = unknown> {
* Optional datasource id to pass while mounting app
*/
dataSourceId?: string;

optionalRef?: Record<string, React.RefObject<HTMLDivElement>>;
}

/**
Expand Down
Loading

0 comments on commit 975d670

Please sign in to comment.