Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Basic layout process nodes and edges #1

Conversation

peluja1012
Copy link
Collaborator

Summary

Renders basic layout of nodes and edges. See https://github.com/elastic/endpoint-app-team/issues/78

image

Checklist

Use strikethroughs to remove checklist items you don't feel are applicable to this PR.

For maintainers

@oatkiller oatkiller force-pushed the resolver-harness-without-endpoint-app-pt2 branch 2 times, most recently from ede44a0 to f046418 Compare January 2, 2020 22:05
@peluja1012 peluja1012 force-pushed the resolver-harness-without-endpoint-app-pt2 branch from b9f1ec3 to b258083 Compare January 6, 2020 16:30
transformOrigin: 'top left',
transform: `translateY(-50%) rotateZ(${angle}rad)`,
};
return <div className={className} style={style} />;
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❓ I think <line x1="${startPosition[0]}" x2="${${startPosition[0]}" y1="${endPosition[0]}" y2="${endPosition[0]}"> would be easier to read and deal with. Forcing these things into HTML/CSS space seems like it makes them a little resistant to some of the things we could do potentially do later to differentiate and polish this product as it matures (e.g. using paint servers for the stroke of the line, filters, masks, maybe animating parts of the transform). @oatkiller I know you said you wanted to defer the conversation until we started "rendering" things - and it looks like that's what we're starting to do here, right? I'm OK if we decide as a team to just use <div>s and <span>s with CSS applied, but I still feel like a system that is specifically built in the Web to handle positioning and rendering things in a Cartesian coordinate system (like we're trying to do) is a better fit here. As fun as it is to see -Math.atan2 in code we use, I think a casual viewer of the code seeing "x1, x2, y1, y2..." could be a little more confident about how it's working and what they change. I acknowledge the downsides are 1) people aren't as familiar with SVG, so maybe that would defeat the whole purpose? 2) IE11 doesn't do CSS transforms in SVG space, so we'd be stuck with it. 3) SVG has more cross-browser issues. But I think the advantage of SVG being vastly more powerful visually and it being easier to "express" the kinds of things we're trying to do in coordinates makes it a better fit for the long game of the Resolver.

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

seems worth investigating but probably out of the scope of this PR (unless that decision would change the graph layout logic.)

If you're interested in using SVGs here, you could make some progress by proving that we can nest any embeddable we want inside of svg?

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

in any case, we'd still have atan and what not. changing a div to a line isn't going to change the math of laying out the graph right?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using the line obviates the need for aTan and other math: You give it coordinates of where it should start and end and it lays itself out.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Which is a case in the overall point I'm making: It's 1 small thing here, but these complexities seem likely to "pile up" as we do more and more along the path of trying to torture CSS flow-based layouts to meet the coordinate-based needs of Resolver.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@oatkiller w.r.t. the question of embedding HTML-based DOM elements in SVG, ForeignObject ( demonstrated here: https://bl.ocks.org/Jverma/2385cb7794d18c51e3ab ) is reported with support across most UAs (with the notable exception of Android for some reason: https://caniuse.com/#feat=mdn-api_svgforeignobjectelement )

*/

// TODO: type root and children
export function* depthFirstPreorder({ root, children }) {
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

now that we have typescript, I stopped putting my arguments in objects like that. typescript can give you hints as to what each is, along w/ inline documentation. also it'll catch you if you're wrong in most situations. plus you can automatically refactor it across the whole project.

Copy link
Owner

@oatkiller oatkiller Jan 6, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

as far as types go, something like this should work. I think the iterator type should be implied.

export function* depthFirstPreorder<T>(root: T, children: (parent: T) => T[]) {

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here's a version that does away with the object wrappers in the arguments and has types:

export function* depthFirstPreorder<T>(root: T, children: (parent: T) => T[]): Iterable<T> {
  const nodesToVisit = [root];
  let currentNode
  while (currentNode = nodesToVisit.shift()) {
    nodesToVisit.unshift(...(children(currentNode) || []));
    yield currentNode;
  }
}

export function* levelOrder<T>(root: T, children: (parent: T) => T[]): Iterable<T> {
  let level = [root];
  while (level.length !== 0) {
    let nextLevel = [];
    for (const node of level) {
      yield node;
      nextLevel.push(...(children(node) || []));
    }
    level = nextLevel;
    nextLevel = [];
  }
}

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. Done.

@@ -0,0 +1,302 @@
/*
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we could make this logic easier to read/understand once we've added some tests. Maybe we do it like:

  1. port it over (you did this)
  2. test it
  3. refactor it until its clean and nice while keeping tests passing

const distanceBetweenNodesInUnits = 1;
const distanceBetweenNodes = distanceBetweenNodesInUnits * unit;

function dataSelector(state: ResolverState) {
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Check out this comment I made on Paul's PR: https://github.com/elastic/kibana/pull/53906/files#r363367614
I explain my ideas around organizing selectors. Can we do that here? Basic plan:

  1. Use a reducer just for this concern, let's call it dataReducer for this example. It takes DataState and ResolverAction and returns a DataState. This reducer doesn't know anything about global state.
  2. Have 'data' selectors that take DataState. They don't know anything about global state. These can be called by the dataReducer since the data reducer takes DataState and these selectors take DataState. These are sorta like model methods for dataState
  3. Import and re-expose some data selectors from store/selectors. The selectors in store/selectors take Global state and are the only selectors used by react components. This is because react components receive global state from useSelector, and they should have no knowledge of the internals of the global state. Since the 'data' selectors take 'DataState' and not global state, you will need to create a version of your selectors that takes GlobalState and then calls the underlying data selectors with the right piece of it. I'm using a helper i made called composeSelectors. Here it is in my PR: https://github.com/elastic/kibana/pull/53619/files#diff-7dd777f1d03cd310741b16bdfb94291fR28
  4. Use only the selectors in store/selectors from react components. These selectors can be thought of as 'public' or you could think of them as model methods for GlobalState.
  5. If some business logic isn't really specific to any particular state, but instead operates on a well defined business construct (e.g. Event, ProcessEvent, User) then place that logic in models/whatever. These types of functions can be pure (they don't rely on state per say, just an entity that could be found anywhere.) They should be easy to test as well, without state there is no need for redux/actions/etc.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

}) => {
const projectionMatrix = useSelector(selectors.projectionMatrix);
const [left, top] = applyMatrix3(startPosition, projectionMatrix);
const length = distance(startPosition[0], startPosition[1], endPosition[0], endPosition[1]);
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the readability of this stanza might be improved if Vector2 had a distance and an angle method.

import { angle, distance } from '../lib/vector2'
// ...
const length = distance(startPosition, endPosition)
const angle = angle(startPosition, endPosition)
// ...

}: {
className?: string;
worldPosition: Vector2;
processEvent: any;
Copy link
Owner

@oatkiller oatkiller Jan 6, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you change this to unknown or some more specific type? With the introduction of unknown and never, we should be able to avoid any (in 99.999% of cases. its an open debate in typescript if a third generic type is needed haha)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed it to ProcessEvent type.

}
}

export function* levelOrder({ root, children }) {
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These look good, but I think having "nodes" in the name of the function would make it easier to understand (like nodesByLevelOrder or levelOrderNodes)

const distanceBetweenNodes = distanceBetweenNodesInUnits * unit;

function dataSelector(state: ResolverState) {
return sampleData.data.result.search_results;
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❓ This is just a placeholder while it gets developed, right?

return sampleData.data.result.search_results;
}

export function isGraphableProcess(event) {
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❓ If we ever get to the point where we want to graph things other than processes, we could change the name of this.


export function eventType(event) {
const {
data_buffer: { event_type_full: type, event_subtype_full: subType },
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❓ maybe call them eventType and eventSubtype to keep things tidy

return event.data_buffer.source_id;
}

function yHalfWayBetweenSourceAndTarget(sourcePosition: Vector2, targetPosition: Vector2) {
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❓ For readability, explaining some of these terms like "source" and "target" would make understanding this for a new reader easier

};
}
);
const widthOfProcessSubtrees = createSelector(
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❓ In comments, some explanation of why you are doing this would be helpful to the reader.


const unit = 100;
const distanceBetweenNodesInUnits = 1;
const distanceBetweenNodes = distanceBetweenNodesInUnits * unit;
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❓ Have you tested making this more than 1? Some ideas Lindsay and I are working on use space from a node's "descending" edge-line to lay out information. Would be super cool if this works (I assume it does).

Copy link

@bkimmel bkimmel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice. Other than the "big question" of how we want to render these things (in HTML or SVG space) and being more descriptive with better names and comments in a coupe places, this looks to be in order.

@oatkiller oatkiller force-pushed the resolver-harness-without-endpoint-app-pt2 branch 2 times, most recently from 2ce5496 to 65c88cd Compare January 7, 2020 22:10
@oatkiller oatkiller force-pushed the resolver-harness-without-endpoint-app-pt2 branch from 0a2e8da to bee753e Compare January 8, 2020 15:30
@peluja1012 peluja1012 force-pushed the resolver-harness-without-endpoint-app-pt2 branch from b258083 to 3bda8d5 Compare January 8, 2020 16:18
* you may not use this file except in compliance with the Elastic License.
*/

export function* depthFirstPreorder<T>(root: T, children: (parent: T) => T[]): Iterable<T> {
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would you mind putting comments on all exported stuff? For example you could write:

/**
 * Sequences a tree, yielding children returned by the `children` function. Sequencing is done in 'depth first preorder' fashion. See https://en.wikipedia.org/wiki/Tree_traversal#Pre-order_(NLR)
 */
export function* depthFirstPreorder<T>(root: T, children: (parent: T) => T[]): Iterable<T> {

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done


interface ServerReturnedResolverData {
readonly type: 'serverReturnedResolverData';
readonly payload: Record<string, any>;
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

want to pair on this?

} from './selectors';

describe('resolver graph layout', () => {
let store: Store<DataState, AnyAction>;
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why not ResolverAction here?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed it to DataAction. I forgot to change it after creating the DataAction type.

* H
*
*/
const processA = {
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These values won't be redefined before each test. Maybe have let processA, processB, etc in the describe function body and then processA = {...}; processB = { ...} in a beforeEach so they are redefined?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good call. Done.

@oatkiller oatkiller force-pushed the resolver-harness-without-endpoint-app-pt2 branch 3 times, most recently from c4ce449 to a02a0d4 Compare January 9, 2020 19:52
@peluja1012
Copy link
Collaborator Author

Closing this PR. This code is not included in the following PR: elastic#53619.

@peluja1012 peluja1012 closed this Jan 13, 2020
elasticmachine pushed a commit that referenced this pull request Mar 9, 2020
* add authRequred: 'optional'

* expose auth status via request context

* update security plugin to use notHandled auth outcome

* capabilities service uses optional auth

* update tests

* attach security headers only to unauthorised response

* add isAuthenticated tests for 'optional' auth mode

* security plugin relies on http.auth.isAuthenticated to calc capabilities

* generate docs

* reword test suit names

* update tests

* update test checking isAuth on optional auth path

* address Oleg comments

* add test for auth: try

* fix

* pass isAuthenticted as boolean via context

* remove response header from notHandled

* update docs

* add redirected for auth interceptor

* security plugin uses t.redirected to be compat with auth: optional

* update docs

* require location header in the interface

* address comments #1

* declare isAuthenticated on KibanaRequest

* remove auth.isAuthenticated from scope

* update docs

* remove unnecessary comment

* do not fail on FakrRequest

* small improvements
oatkiller pushed a commit that referenced this pull request May 6, 2020
…lastic#65159)

* Remove stray rules that should've been deleted
* Update rule.ts and tests
* Remove deleted prebuilt rules from cypress ES archive (#1)
oatkiller pushed a commit that referenced this pull request May 6, 2020
* Remove stray rules that should've been deleted
* Update rule.ts and tests
* Remove deleted prebuilt rules from cypress ES archive (#1)
oatkiller pushed a commit that referenced this pull request Sep 21, 2020
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
oatkiller pushed a commit that referenced this pull request Sep 21, 2020
…lastic#77990)

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
oatkiller pushed a commit that referenced this pull request Jun 2, 2021
…-security-solution-es-utils package (elastic#99828)

## Summary

Fixes the hopefully last circular dependency issues between security solutions and lists.

* Adds a package of `@kbn/securitysolution-es-utils` and moves files from security solutions into that package.
* Re-ingests that package back into lists 

Before this PR if you ran:

```ts
node scripts/find_plugins_with_circular_deps.js --debug
```

Then you would get:

```
 debg !!!!!!!!!!!!!! CIRCULAR DEPENDENCIES FOUND !!!!!!!!!!!!!!
      !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
      ! Circular dependencies were found, you can find below  !
      ! all the paths involved.                               !
      !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
 debg   01) x-pack/plugins/lists/server/index.ts -> x-pack/plugins/lists/server/plugin.ts -> x-pack/plugins/lists/server/routes/init_routes.ts -> x-pack/plugins/lists/server/types.ts -> x-pack/plugins/lists/server/services/lists/list_client.ts -> x-pack/plugins/lists/server/siem_server_deps.ts -> x-pack/plugins/security_solution/server/index.ts -> x-pack/plugins/security_solution/server/plugin.ts
        02) x-pack/plugins/lists/server/index.ts -> x-pack/plugins/lists/server/plugin.ts -> x-pack/plugins/lists/server/routes/init_routes.ts -> x-pack/plugins/lists/server/types.ts -> x-pack/plugins/lists/server/services/lists/list_client.ts -> x-pack/plugins/lists/server/siem_server_deps.ts -> x-pack/plugins/security_solution/server/index.ts -> x-pack/plugins/security_solution/server/plugin.ts -> x-pack/plugins/security_solution/server/lib/compose/kibana.ts -> x-pack/plugins/security_solution/server/lib/framework/kibana_framework_adapter.ts -> x-pack/plugins/security_solution/server/types.ts
        03) x-pack/plugins/lists/server/index.ts -> x-pack/plugins/lists/server/plugin.ts -> x-pack/plugins/lists/server/routes/init_routes.ts -> x-pack/plugins/lists/server/types.ts -> x-pack/plugins/lists/server/services/lists/list_client.ts -> x-pack/plugins/lists/server/siem_server_deps.ts -> x-pack/plugins/security_solution/server/index.ts -> x-pack/plugins/security_solution/server/plugin.ts -> x-pack/plugins/security_solution/server/lib/compose/kibana.ts -> x-pack/plugins/security_solution/server/endpoint/types.ts -> x-pack/plugins/security_solution/server/endpoint/endpoint_app_context_services.ts
        04) x-pack/plugins/lists/server/index.ts -> x-pack/plugins/lists/server/plugin.ts -> x-pack/plugins/lists/server/routes/init_routes.ts -> x-pack/plugins/lists/server/types.ts -> x-pack/plugins/lists/server/services/lists/list_client.ts -> x-pack/plugins/lists/server/siem_server_deps.ts -> x-pack/plugins/security_solution/server/index.ts -> x-pack/plugins/security_solution/server/plugin.ts -> x-pack/plugins/security_solution/server/lib/compose/kibana.ts -> x-pack/plugins/security_solution/server/endpoint/types.ts -> x-pack/plugins/security_solution/server/endpoint/endpoint_app_context_services.ts -> x-pack/plugins/security_solution/server/fleet_integration/fleet_integration.ts
        05) x-pack/plugins/lists/server/index.ts -> x-pack/plugins/lists/server/plugin.ts -> x-pack/plugins/lists/server/routes/init_routes.ts -> x-pack/plugins/lists/server/types.ts -> x-pack/plugins/lists/server/services/lists/list_client.ts -> x-pack/plugins/lists/server/siem_server_deps.ts -> x-pack/plugins/security_solution/server/index.ts -> x-pack/plugins/security_solution/server/plugin.ts -> x-pack/plugins/security_solution/server/lib/compose/kibana.ts -> x-pack/plugins/security_solution/server/endpoint/types.ts -> x-pack/plugins/security_solution/server/endpoint/endpoint_app_context_services.ts -> x-pack/plugins/security_solution/server/fleet_integration/fleet_integration.ts -> x-pack/plugins/security_solution/server/endpoint/services/index.ts -> x-pack/plugins/security_solution/server/endpoint/services/artifacts/index.ts -> x-pack/plugins/security_solution/server/endpoint/services/artifacts/manifest_manager/index.ts -> x-pack/plugins/security_solution/server/endpoint/services/artifacts/manifest_manager/manifest_manager.ts
        06) x-pack/plugins/lists/server/index.ts -> x-pack/plugins/lists/server/plugin.ts -> x-pack/plugins/lists/server/routes/init_routes.ts -> x-pack/plugins/lists/server/types.ts -> x-pack/plugins/lists/server/services/lists/list_client.ts -> x-pack/plugins/lists/server/siem_server_deps.ts -> x-pack/plugins/security_solution/server/index.ts -> x-pack/plugins/security_solution/server/plugin.ts -> x-pack/plugins/security_solution/server/lib/compose/kibana.ts -> x-pack/plugins/security_solution/server/endpoint/types.ts -> x-pack/plugins/security_solution/server/endpoint/endpoint_app_context_services.ts -> x-pack/plugins/security_solution/server/fleet_integration/fleet_integration.ts -> x-pack/plugins/security_solution/server/endpoint/services/index.ts -> x-pack/plugins/security_solution/server/endpoint/services/artifacts/index.ts -> x-pack/plugins/security_solution/server/endpoint/services/artifacts/manifest_manager/index.ts -> x-pack/plugins/security_solution/server/endpoint/services/artifacts/manifest_manager/manifest_manager.ts -> x-pack/plugins/security_solution/server/endpoint/lib/artifacts/index.ts -> x-pack/plugins/security_solution/server/endpoint/lib/artifacts/lists.ts
        07) x-pack/plugins/lists/server/index.ts -> x-pack/plugins/lists/server/plugin.ts -> x-pack/plugins/lists/server/routes/init_routes.ts -> x-pack/plugins/lists/server/types.ts -> x-pack/plugins/lists/server/services/lists/list_client.ts -> x-pack/plugins/lists/server/siem_server_deps.ts -> x-pack/plugins/security_solution/server/index.ts -> x-pack/plugins/security_solution/server/plugin.ts -> x-pack/plugins/security_solution/server/lib/compose/kibana.ts -> x-pack/plugins/security_solution/server/endpoint/types.ts -> x-pack/plugins/security_solution/server/endpoint/endpoint_app_context_services.ts -> x-pack/plugins/security_solution/server/fleet_integration/fleet_integration.ts -> x-pack/plugins/security_solution/server/fleet_integration/handlers/install_prepackaged_rules.ts
        08) x-pack/plugins/lists/server/index.ts -> x-pack/plugins/lists/server/plugin.ts -> x-pack/plugins/lists/server/routes/init_routes.ts -> x-pack/plugins/lists/server/types.ts -> x-pack/plugins/lists/server/services/lists/list_client.ts -> x-pack/plugins/lists/server/siem_server_deps.ts -> x-pack/plugins/security_solution/server/index.ts -> x-pack/plugins/security_solution/server/plugin.ts -> x-pack/plugins/security_solution/server/lib/compose/kibana.ts -> x-pack/plugins/security_solution/server/endpoint/types.ts -> x-pack/plugins/security_solution/server/endpoint/endpoint_app_context_services.ts -> x-pack/plugins/security_solution/server/fleet_integration/fleet_integration.ts -> x-pack/plugins/security_solution/server/fleet_integration/handlers/install_prepackaged_rules.ts -> x-pack/plugins/security_solution/server/lib/detection_engine/routes/rules/add_prepackaged_rules_route.ts
        09) x-pack/plugins/lists/server/index.ts -> x-pack/plugins/lists/server/plugin.ts -> x-pack/plugins/lists/server/routes/init_routes.ts -> x-pack/plugins/lists/server/types.ts -> x-pack/plugins/lists/server/services/lists/list_client.ts -> x-pack/plugins/lists/server/siem_server_deps.ts -> x-pack/plugins/security_solution/server/index.ts -> x-pack/plugins/security_solution/server/plugin.ts -> x-pack/plugins/security_solution/server/lib/detection_engine/signals/types.ts
        10) x-pack/plugins/lists/server/index.ts -> x-pack/plugins/lists/server/plugin.ts -> x-pack/plugins/lists/server/routes/init_routes.ts -> x-pack/plugins/lists/server/types.ts -> x-pack/plugins/lists/server/services/lists/list_client.ts -> x-pack/plugins/lists/server/siem_server_deps.ts -> x-pack/plugins/security_solution/server/index.ts -> x-pack/plugins/security_solution/server/plugin.ts -> x-pack/plugins/security_solution/server/lib/detection_engine/signals/signal_rule_alert_type.ts -> x-pack/plugins/security_solution/server/lib/detection_engine/signals/utils.ts
        11) x-pack/plugins/lists/server/index.ts -> x-pack/plugins/lists/server/plugin.ts -> x-pack/plugins/lists/server/routes/init_routes.ts -> x-pack/plugins/lists/server/types.ts -> x-pack/plugins/lists/server/services/lists/list_client.ts -> x-pack/plugins/lists/server/siem_server_deps.ts -> x-pack/plugins/security_solution/server/index.ts -> x-pack/plugins/security_solution/server/plugin.ts -> x-pack/plugins/security_solution/server/lib/detection_engine/signals/signal_rule_alert_type.ts -> x-pack/plugins/security_solution/server/lib/detection_engine/signals/executors/query.ts
        12) x-pack/plugins/lists/server/index.ts -> x-pack/plugins/lists/server/plugin.ts -> x-pack/plugins/lists/server/routes/init_routes.ts -> x-pack/plugins/lists/server/types.ts -> x-pack/plugins/lists/server/services/lists/list_client.ts -> x-pack/plugins/lists/server/siem_server_deps.ts -> x-pack/plugins/security_solution/server/index.ts -> x-pack/plugins/security_solution/server/plugin.ts -> x-pack/plugins/security_solution/server/lib/detection_engine/signals/signal_rule_alert_type.ts -> x-pack/plugins/security_solution/server/lib/detection_engine/signals/executors/query.ts -> x-pack/plugins/security_solution/server/lib/detection_engine/signals/search_after_bulk_create.ts -> x-pack/plugins/security_solution/server/lib/detection_engine/signals/filters/filter_events_against_list.ts -> x-pack/plugins/security_solution/server/lib/detection_engine/signals/filters/types.ts
        13) x-pack/plugins/lists/server/index.ts -> x-pack/plugins/lists/server/plugin.ts -> x-pack/plugins/lists/server/routes/init_routes.ts -> x-pack/plugins/lists/server/types.ts -> x-pack/plugins/lists/server/services/lists/list_client.ts -> x-pack/plugins/lists/server/siem_server_deps.ts -> x-pack/plugins/security_solution/server/index.ts -> x-pack/plugins/security_solution/server/plugin.ts -> x-pack/plugins/security_solution/server/lib/detection_engine/signals/signal_rule_alert_type.ts -> x-pack/plugins/security_solution/server/lib/detection_engine/signals/executors/threat_match.ts
        14) x-pack/plugins/lists/server/index.ts -> x-pack/plugins/lists/server/plugin.ts -> x-pack/plugins/lists/server/routes/init_routes.ts -> x-pack/plugins/lists/server/types.ts -> x-pack/plugins/lists/server/services/lists/list_client.ts -> x-pack/plugins/lists/server/siem_server_deps.ts -> x-pack/plugins/security_solution/server/index.ts -> x-pack/plugins/security_solution/server/plugin.ts -> x-pack/plugins/security_solution/server/lib/detection_engine/signals/signal_rule_alert_type.ts -> x-pack/plugins/security_solution/server/lib/detection_engine/signals/executors/threat_match.ts -> x-pack/plugins/security_solution/server/lib/detection_engine/signals/threat_mapping/create_threat_signals.ts -> x-pack/plugins/security_solution/server/lib/detection_engine/signals/threat_mapping/get_threat_list.ts -> x-pack/plugins/security_solution/server/lib/detection_engine/signals/threat_mapping/types.ts
        15) x-pack/plugins/lists/server/index.ts -> x-pack/plugins/lists/server/plugin.ts -> x-pack/plugins/lists/server/routes/init_routes.ts -> x-pack/plugins/lists/server/types.ts -> x-pack/plugins/lists/server/services/lists/list_client.ts -> x-pack/plugins/lists/server/siem_server_deps.ts -> x-pack/plugins/security_solution/server/index.ts -> x-pack/plugins/security_solution/server/plugin.ts -> x-pack/plugins/security_solution/server/lib/detection_engine/signals/signal_rule_alert_type.ts -> x-pack/plugins/security_solution/server/lib/detection_engine/signals/executors/ml.ts
        16) x-pack/plugins/lists/server/index.ts -> x-pack/plugins/lists/server/plugin.ts -> x-pack/plugins/lists/server/routes/init_routes.ts -> x-pack/plugins/lists/server/types.ts -> x-pack/plugins/lists/server/services/lists/list_client.ts -> x-pack/plugins/lists/server/siem_server_deps.ts -> x-pack/plugins/security_solution/server/index.ts -> x-pack/plugins/security_solution/server/plugin.ts -> x-pack/plugins/security_solution/server/endpoint/routes/trusted_apps/index.ts -> x-pack/plugins/security_solution/server/endpoint/routes/trusted_apps/handlers.ts
        17) x-pack/plugins/lists/server/index.ts -> x-pack/plugins/lists/server/plugin.ts -> x-pack/plugins/lists/server/routes/init_routes.ts -> x-pack/plugins/lists/server/types.ts -> x-pack/plugins/lists/server/services/lists/list_client.ts -> x-pack/plugins/lists/server/siem_server_deps.ts -> x-pack/plugins/security_solution/server/index.ts -> x-pack/plugins/security_solution/server/plugin.ts -> x-pack/plugins/security_solution/server/endpoint/routes/trusted_apps/index.ts -> x-pack/plugins/security_solution/server/endpoint/routes/trusted_apps/handlers.ts -> x-pack/plugins/security_solution/server/endpoint/routes/trusted_apps/service.ts
        18) x-pack/plugins/lists/server/index.ts -> x-pack/plugins/lists/server/plugin.ts -> x-pack/plugins/lists/server/routes/init_routes.ts -> x-pack/plugins/lists/server/types.ts -> x-pack/plugins/lists/server/services/lists/list_client.ts -> x-pack/plugins/lists/server/siem_server_deps.ts -> x-pack/plugins/security_solution/server/index.ts -> x-pack/plugins/security_solution/server/plugin.ts -> x-pack/plugins/security_solution/server/endpoint/routes/trusted_apps/index.ts -> x-pack/plugins/security_solution/server/endpoint/routes/trusted_apps/handlers.ts -> x-pack/plugins/security_solution/server/endpoint/routes/trusted_apps/service.ts -> x-pack/plugins/security_solution/server/endpoint/routes/trusted_apps/mapping.ts

 debg !!!!!!!!!!!!!!!!! UP TO DATE ALLOWED LIST !!!!!!!!!!!!!!!!!!
      !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
      ! The declared circular dependencies allowed list is up    !
      ! to date and includes every plugin listed in above paths. !
      !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

      The allowed circular dependencies list is (elastic#3):
      'x-pack/plugins/lists -> x-pack/plugins/security_solution',
 succ None non allowed circular dependencies were found
```

Now you get:

```
  debg !!!!!!!!!!!!!!!!! UP TO DATE ALLOWED LIST !!!!!!!!!!!!!!!!!!
      !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
      ! The declared circular dependencies allowed list is up    !
      ! to date and includes every plugin listed in above paths. !
      !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

      The allowed circular dependencies list is (#1):
      'x-pack/plugins/lists -> x-pack/plugins/security_solution',
 succ None non allowed circular dependencies were found
```

### Checklist

Delete any items that are not applicable to this PR.

- [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
oatkiller pushed a commit that referenced this pull request Mar 28, 2022
…ic#126751)

* Client side execution app level context propagation

* context$ + apm rum integration

* invert the context parent \ child relationship (cc @mikhail)
move more things to top level context

* Pass down context to apm on server

* types

* eslint

* parent <> child

* docs + eslint + jest

* execution context mock

* eslint

* jest

* jest

* server jest

* check

* jest

* storybook

* jest

* report the current space

* fix server side context container

* Remove spaces for now

* docssss

* jest

* lint

* test

* docs

* revert file

* doc

* all context params are optional

* clear on page change

* lint

* ts

* skipped test again

* testing fixes

* oops

* code review #1

* code review elastic#2

* getAsLabels

* maps inherit dashboard context

* docs

* ts

* Give common context to all vis editors

* fix test

* ts \ es \ tests

* labels

* missing types

* docsy docs

* cr elastic#3

* improve jest

* Use editor name

* Update src/plugins/visualizations/public/visualize_app/components/visualize_editor.tsx

Co-authored-by: Marco Liberati <dej611@users.noreply.github.com>

* fix maps context

* jest tests for maps

* cr

* docs

* Update execution_context.test.ts

* docs

* lint

Co-authored-by: Marco Liberati <dej611@users.noreply.github.com>
(cherry picked from commit d5416ed)

# Conflicts:
#	src/plugins/discover/public/application/context/context_app.tsx
#	x-pack/plugins/lens/public/app_plugin/app.tsx
oatkiller pushed a commit that referenced this pull request Mar 28, 2022
* Client side execution app level context propagation

* context$ + apm rum integration

* invert the context parent \ child relationship (cc @mikhail)
move more things to top level context

* Pass down context to apm on server

* types

* eslint

* parent <> child

* docs + eslint + jest

* execution context mock

* eslint

* jest

* jest

* server jest

* check

* jest

* storybook

* jest

* report the current space

* fix server side context container

* Remove spaces for now

* docssss

* jest

* lint

* test

* docs

* revert file

* doc

* all context params are optional

* clear on page change

* lint

* ts

* skipped test again

* testing fixes

* oops

* code review #1

* code review elastic#2

* getAsLabels

* maps inherit dashboard context

* docs

* ts

* Give common context to all vis editors

* fix test

* ts \ es \ tests

* labels

* missing types

* docsy docs

* cr elastic#3

* improve jest

* Use editor name

* Update src/plugins/visualizations/public/visualize_app/components/visualize_editor.tsx

Co-authored-by: Marco Liberati <dej611@users.noreply.github.com>

* fix maps context

* jest tests for maps

* cr

* docs

* Update execution_context.test.ts

* docs

* lint

Co-authored-by: Marco Liberati <dej611@users.noreply.github.com>
kibanamachine added a commit that referenced this pull request Dec 6, 2023
## Summary

### This PR enables user roles testing in FTR

We use SAML authentication to get session cookie for user with the
specific role. The cookie is cached on FTR service side so we only make
SAML auth one time per user within FTR config run. For Kibana CI service
relies on changes coming in elastic#170852

In order to run FTR tests locally against existing MKI project:
- add `.ftr/role_users.json` in Kibana root dir
```
{
  "viewer": {
    "email": "...",
    "password": "..."
  },
  "developer": {
    "email": "...",
    "password": "..."
  }
}

```
- set Cloud hostname (!not project hostname!) with TEST_CLOUD_HOST_NAME,
e.g.
`export TEST_CLOUD_HOST_NAME=console.qa.cld.elstc.co`


### How to use:

- functional tests:
```
const svlCommonPage = getPageObject('svlCommonPage');

before(async () => {
  // login with Viewer role  
  await svlCommonPage.loginWithRole('viewer');
  // you are logged in in browser and on project home page, start the test 
});

it('has project header', async () => {
  await svlCommonPage.assertProjectHeaderExists();
});
```

- API integration tests:
```
const svlUserManager = getService('svlUserManager');
const supertestWithoutAuth = getService('supertestWithoutAuth');
let credentials: { Cookie: string };

before(async () => {
  // get auth header for Viewer role  
 credentials = await svlUserManager.getApiCredentialsForRole('viewer');
});

it('returns full status payload for authenticated request', async () => {
    const { body } = await supertestWithoutAuth
    .get('/api/status')
    .set(credentials)
    .set('kbn-xsrf', 'kibana');

    expect(body.name).to.be.a('string');
    expect(body.uuid).to.be.a('string');
    expect(body.version.number).to.be.a('string');
});
```

Flaky-test-runner: 

#1
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/4081
elastic#2
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/4114

---------

Co-authored-by: Robert Oskamp <traeluki@gmail.com>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Aleh Zasypkin <aleh.zasypkin@gmail.com>
oatkiller pushed a commit that referenced this pull request Jun 20, 2024
## Summary
Set `security.session.cleanupInterval` to 5h for session concurrency
test.

### **Prerequisites**

- Task for session cleanup with [default schedule set to
1h](https://github.com/elastic/kibana/blob/main/x-pack/plugins/security/server/config.ts#L222).
- Task polling interval is set to
[3000ms](https://github.com/elastic/kibana/blob/main/x-pack/plugins/task_manager/server/config.ts#L13).
- We override `scheduledAt` once we make a request in
[runCleanupTaskSoon](https://github.com/elastic/kibana/blob/main/x-pack/test/security_api_integration/tests/session_concurrent_limit/cleanup.ts#L145).

### **Hypothesis**

Taking into consideration that:

- `session_cleanup` task is not the only one scheduled during test run.
- There is sort of an exponential backoff implemented for task polling
if there are too many retries.
- Clock jitter.

I had a hypothesis that if our whole test run exceeds 1h or polling
interval gets adjusted because of retries we might end up executing the
scheduled cleanup before we trigger `runCleanupTaskSoon` (this is there
we drop 1 session already).

### **FTR runs (x55 each)**

- `cleanupInterval` set to 5h:
[#1](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/5986)
:green_circle:,
[elastic#2](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/5987)
:green_circle:
- `cleanupInterval` set to default 1h:
[#1](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/5983)
:green_circle:,
[elastic#2](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/5982)
:red_circle: (has 2 failures out of 55)


### Checklist

- [x] [Flaky Test
Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was
used on any tests changed

### For maintainers

- [x] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)

__Fixes: https://github.com/elastic/kibana/issues/149091__
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants