Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
* feat(fleet): fix label names, change latency from seconds to microseconds and add missing ownerIDs * feat(fleet): add label group_id to service metrics * feat(fleet): approach to improve getting ownerID for service metrics * doc(fleet): update fleet service openapi spec * doc(fleet): small grammar correction * Added integration tests in python for agents (#559) Co-authored-by: Stéfano de Faria <sfaria@daitan.com> * wip * wip(orb-ui): able to display contract * wip(orb-ui): testing with env var * added agent groups feature test (#568) * added agent groups feature test * feat(orb-ui): read with dotenv the following properties from .env `PS_SID` `PS_GROUP_KEY` * feat(orb-ui): do not track further modifications to environment.env.ts, which is overwritten by npm task config:env * feat(orb-ui): spread envVars to app environment object * fix .gitignore and missing coma * 556 (#582) * added agent groups feature test * fix(orb-ui): fix show filter configurations for handlers * todo: look for latest versions of configuration definitions * wip * feat(otel): wip receiver / exporter * feat(orb-ui): suggest name for handlers * fix(orb-ui): reset form when backstep * Fixed recipient value in the password recovery email template (#593) * feat(otel): wip functional prom receiver / prom exporter on orb agent * feat(otel): wip functional prom receiver / prom exporter on orb agent * feat(fleet): add identify to service metrics implentation * feat(orb-ui): re-enable profile page * feat(otel): wip otel reciver on orb agent * feat(orb-ui): remove undesired fields from formpage * 557 - python api policies test (#587) * added integration tests in python for policies * feat(orb-ui): Adopt fullName instead of firstName & lastName * feat(orb-agent) wip create pktvisor receiver component * feat(otel): wip pktvisor otel receiver * feat(otel): wip move logic to pktvisor implementation * feat(otel): wip use pktvisor otel factory * feat(otel): wip use pktvisor otel factory * test(otel): fix unit and integration tests * feat(otel): fix remove go.mod unnecessary dependencies * feat(orb-ui): change layout to two cards in a row, fields per row per card. * fix(sinks): fix status code response of sink ceration * feat(otel): organize folder structure to pktvisor otel receiver * feat(otel): Fix load config unit test * fix(orb-ui): Missing trailling space in scss file * fix(orb-ui): fix username property missing * feat(otel): Fix orb-agent compilation fail * fix(orb-ui): send/retrieve company and fullName using metadata field * test(otel): wip setting up integration tests with pktvisor container * test(otel): crate pktvisor docker config * test(otel): wip making basic pktvisor config works * fix(sinks): reformat code * feat(orb-ui): Add reset-password page and move request-password component to proper folder * fix(orb-ui): sink list loading bar always reloading * fix(orb-ui): pages never stop reloading * fix for agent groups list page * fix for agents list page * fix for datasets list page * fix for agent policies list page * feature(sinks): add service metrics to api * 558 dataset tests (#597) * added agent groups feature test * fix: breadcrumb 'Home' to 'User Profile' * test(otel): wip make pktvisor container run (but still not scrapping) * test(otel): wip integration test pktvisor container * feat(policies): add service metrics to api * test(otel): wip integration test pktvisor * feature(sinks): improvements on service metrics * test(otel): fix integration tests with pktvisor (problems exposing the container localhost) * test(otel) minor corrections * fix(orb-ui): fix display of agent tags on agent group details view * fix(orb-ui): fix line break on third step - misaligned select field * fix(orb-ui): remove dataset view 'tags' field * fix(orb-ui): remove required * from key/value tag pair in sink add/edit pages * fix(orb-ui): remove required * from sink details view tag field * feat(otel-exporter): bootstraping otlp exporter for orb agent * fix(orb-ui): Remove red asterisk '*' required tags for agents in view and details page * test(otel-exporter): create unit test to otel exporter config * fix(orb-ui): fix error msg * fix(orb-ui): Remove required asterisk '*' from version field in agent policy detail view * feat(otel): create a config to decide which way to scrape metrics * fix(orb-ui): fix not displaying agent matches on creation * feat(otel): code review changes * fix(orb-ui): fix menu items order in navigation menu * feat(otel): code review changes * fix(orb-ui): fix agent view copy command button not changing icon after click * test(otel): create unit test to otlp exporter * test(otel): using otlp exporter on orb agent * fix(orb-ui): #625 fix order on dashboard page (#637) * fix(orb-ui): fix order of orb entities on dashboard page * fix(orb-ui): fix wrong row ordering * english correction * feat(otel): wip code sanitization * feat(otel): wip extract code to new methods * feat(otel): Create otlp exporter starter on pktvisor startup. * enhancement(orb-ui): improve home page card ordering for user actions * fix(dataset): Fix http patch to put on edition. * feat(otel): Make otel configuration parameters more readable when configuring pktvisor * bump to 0.11.0-develop * feat(otel-exporter): Fix merge conflicts with ui * fix ui conflicts * fix missing PKTVISOR_TAG in GitHub actions. bring go builds up to 1.17 (#664) * fix missing PKTVISOR_TAG in GitHub actions * bring go builds up to 1.17 * add agent README * test(sinks): increase unit test coverage at data access layer * test(sinks): increase service unit test coverage * test(sinks): increase endpoint unit tests coverage * add dnstap input (#669) * fix(orb-ui): fix button outside form scope not submitting forgotPassword fix(orb-ui): override nbResetPassword resetPassword method * feat(otel-exporter): create otel otlp exporter starter * test(sinks): increase endpoint unit test coverage * bug(sinker): Applying changes from pktvisor/types.go to sinker/pktvisor/types.go to fix prom particle generation * test(sinks): fix test cases description * fix(orb-ui): remove option to choose backend type for agent policy * fix(orb-ui): tap & input of agent policy fix: use input predefined in tap configuration fix: show input in dropdown selector for tap fix: fail gracefully - add label stating there are no taps listed for when there are no agents running. Links to Agents List Page. * fix(orb-ui): fix handler configuration json not being posted * bug(sinker): wip sinker console output * bug(sinker): wip remove unused console output * bug(sinker): wip ugly way to solve the topGeoLoc and topASN problem * testing agents through the ui (#667) * testing agents through the ui * bug(sinker): wip solving TopGeoLoc using regex * test(sinks): fix postgres unit test description and fix error return for update non-existing sink * test(sinks): fix expected error of postgres unit test cases * add the indirect go mods, ensure static link during CGO build. * bug(sinker): wip TopGeoLoc * bug(sinker): wip using regex to get exceptions to prometheus conversion * bug(sinker): return version to unknown on version.go file * bug(sinker): fixing go.mod file * fix(orb-ui): Page not responding after clicking on Register nor logging into dashboard * feat(sinker): remove unnecessary comments and console writers * feat(orb-ui): add advanced options to tap configuration and filter configuration in agent policies. * feat(fleet): remove last_db_data and agent_metadata from agent list endpoint * feat(orb-ui): mark text where user could/should change interface * feat(orb-ui): * remove odd <hr/> breaks * add labels to each section of input/tap configuration * feat(orb-ui): change color for marked text * remove codeql analysis for now * feat(orb-agent): extract pktvisor otel receiver / exporter functions to a new file * feat(orb-agent): Remove policy from agent local repo on removal action * 679 second policy (#683) * apply two policies to an agent Co-authored-by: Stéfano de Faria <sfaria@daitan.com> * fix ui conflicts * fix conflicts with python-test * fix(orb-ui): chain observables for dataset related entities and display name of such entities when retrieved * fix(orb-ui): Fix wrong spacing between sink display and elements above it * fix(orb-ui): keep sinks list instead of only their names * refactor(orb-ui): cleanup unused imports and code style * refactor(orb-ui): implement ngOnDestroy to unsubscribe from observables * fix(orb-ui): remove option to filter dataset by tags * recovering some changes that I noticed that shold be on previous PR (#724) * refactor(orb-ui): refactor pactsafe integration * refactor(orb-ui): finish integration with pactsafe * show only checkbox with 'agreement' label * remove minified pactsafe wrapper * contract is now shown when user clicks link * int_tests - automate account registration with credentials required (email and password) if they are not yet registered (#726) * register an account using email and password if credentials are not registered yet * fix(orb-ui): Policy Details View (#734) * wip * fix(orb-ui): Add more details on Agent Policy Details View * add handlers section to display name of each handler * add input type and tap name section * feat(orb-ui): list handlers as accordion items * show handler details inside accordion body, represented in JSON * Feat/orb agent debug arg (#728) * feat(orb-agent): Create debug parameters to agent * feat(orb-agent): Use right debug configuration args * fix(orb-ui): #737 register button not working (#740) * fix(orb-ui): register button not working * fix(orb-ui): enable sandbox when not production build * fix(orb-ui): register button not working - _ps module not available * Bug/mqtt incoming stoped error (#742) * feat(orb-ui): add advanced options to tap configuration and filter configuration in agent policies. * feat(orb-ui): * remove odd <hr/> breaks * add labels to each section of input/tap configuration * recovering some changes that I noticed that shold be on previous PR (#724) * fix - upgrade paho.mqtt version and setup auto reconnect configuration * remove ui e python-test * merge fix with ui and python-test * bug(orb-agent) Change keep alive and ping timeout configuration for bigger times Co-authored-by: gpazuch <1490938+gpazuch@users.noreply.github.com> Co-authored-by: manrodrigues <78241475+manrodrigues@users.noreply.github.com> * fix(orb-ui): disable ps contract box if not prod (#741) * fix(orb-ui): register button not working * fix(orb-ui): enable sandbox when not production build * fix(orb-ui): register button not working - _ps module not available * fix(orb-ui): hide contract box when development and not production * fix(orb-ui): remove unused imports (#744) * feat(orb-ui): Display warning when no tags added for agent (#735) * Show message in agent add and agent list page warning user that agent has no tags to it. * Also fix string for tag color on sink page * Automated tests for delete a policy applied to an agent (#743) Automated tests for delete a policy applied to an agent * fix(orb-ui): Add missing required asterisk (*) on handler label field (#753) * doc(policy): update dataset and agent policy creation request (#635) * Feat/orb policy associate schema version (#732) * feat(orb-agent): Create schema version field on policy and receive on policy creation * feat(orb-agent): Update openapi spec to reflect schema version new field * feat(fleet): Create schema version on fleet backends return * feat(policy): Changes and enhancements after code review * Bug/orb sinker ipv6 metrics (#755) * bug(sinker): change colon for semicolon when creating prom labels (avoid problem of ipv6 hex names, which has colons) * bug(sinker) change split method on label creation * bug(sinker): improve log messages when prometheus components fail to be created * bug(sinker): change log message type to error * bug(sinker): change colon for semicolon on quatile lable * delete dataset tests (#759) * delete dataset tests * Tests/login (#761) * automated tests for login * fix(orb-ui): add 'e.g.: ' to examples (#757) * fix(orb-ui): fix error message during register (#758) * authentication module no longer expects valid token on /users endpoint * remove code for redirection on /users endpoint response - force auth * fix(orb-ui): fix agent policy not able to apply when filter is empty (#762) * fix(orb-ui): expose pactsafe env vars to docker space in local builds (#770) * fix(orb-ui): expose pactsafe env vars to docker space in local builds * building orb-ui via `make ui` -> make sure PS_SID and PS_GROUP_KEY are present in the environment. * Fix whitespace * fix(orb-ui): Fix environment setup When reading from environment, setenv.js would check if property is available, and determine wheter to produce the output to the environment file supplied to Angular App. Since Dockerfile was patched to pass down such variables, they're now always present in the docker env., though set as empty string ''. Setenv.js check changed to string comparisson. * fix(orb-ui): check only that PS can be enabled, no longer check for prod * refactor(orb-agent): enhancement on log messages (#750) * refactor(orb-agent): enhancement on log messages Create more wrappers for errors messages related to mqtt errors * feat(orb-agent): remove unnecessary param version on agent creation * Enhancement on logs for orb-agent policy removal * security(org-agent): remove from logs key which is used to log on mqtt * Fix policies migrations error (#774) * Fix migrations error setting valid value for schema_version column creation * bug(policies) make dafault value for schema_version 1.0 * Bug/orb mqtt agent group order creation (#772) * wip investigation of the problem * wip to solve bug on agent group order creation * Wip create connection with agent group on agent creation * Remove docker-compose auth changes which is unrelated this feature * Fix unit test on agent view (missing mock implemetation for RetrieveAllByAgent method) * bug(fleet): logs improvements and removal of unnecessary commets * Change agent_group_id to group_id to keep consistency * Handle error for retrieve agent group when connect agents to group channel * New version 0.12.0-develop (#777) * changing is_credentials_registered variable after register the account (#786) * get correct agent through UI (#787) * Remove orb_tag validation on agent edition (#789) * change title of prometheus configuration (#790) * Feat/orb agent handle multiple datasets (#781) * feat(orb-agent): make orb-agent handle with multiple datasets for the same policy (avoid apply policy multiple times) * initial logic for updating a policy * Create logic to handle policy updated based on version * create a timeout parameter on pktvisor request method * Enhancements after code review * removing duplicated tests (#794) * automated tests: covering integration scenarios (#799) * automated tests: covering integration scenarios * Feature/orb policies increase unit tests coverage (#782) * test(policies): improvements on unit tests coverage of data acess layer * test(policies): improvements on service unit test coverage * test(policies): improvements on unit test coverage of policies API * Orb - tests documentation (#785) * orb_int_tests_description * Feat/orb policy last modify (#800) * Fix schema_version empty valeu * create last_modified field to registry the date of policy edition * create last modified field on endpoint response * Refactor/orb sinker change top k metrics label (#795) * wip refactor prom particle label for top k metrics * minor corrections * create sinker parser for top n metrics * remove unnecessary console prints * changes requested on code review * Bootstraping unit tests to sinker process metrics | mapped dhcp metrics * Enhancements on unit tests * remove migrations from this PR * Tests/integration tests (#803) * automated tests: covering integration scenarios * fix(orb-ui): fix advancedOptions filter (#783) * fix filter advancedOptions, enabling correct rendering of advanced and non-advanced options for agent policies * Feat/orb agent reset (#816) * wip orb-agent reset logic * wip create endpoint to receive orb-agent reset request * create logic for backend reset enhancements on logs for backend reset procedure create comms for receive agent reset message * rollback build info version to unknown * changes after code review * build debug agent in ci * Avoiding NaN on prometheus sending metrics every 1 minute even when its 0 (#818) * set pktvisor port test (#823) * (hotfix): node/types specify version 12.20.42 - 12.20.43 breaks other deps (#824) * Fix dataset overriting to empty when edit a policy (#826) * Feat/orb policy last modify (#819) * Fix schema_version empty valeu * create last_modified field to registry the date of policy edition * create last modified field on endpoint response * Add last modified field for list policies endpoint * fix typo on agent tags help message (#834) * add undersocre bofore ASN (#835) * Bug/orb agent tag edition (#828) * Fix dataset overriting to empty when edit a policy * wip solve orb-agent tag edition problem * enhancements on logs for group unsubscrition * wip tags edition * code sanitization * fix group info struct name * changes after code review * fix(orb-ui): (#837) * require two names, space separated, for full user name * send full user name as signer id when contract agreement * pactsafe confirmation email set to true - user should receive a copy of contract * fix(orb-ui): (#838) * agent provisioning command to use 'develop' tagged docker image -> ns1labs/orb-agent:develop * creating unit test and fixing topASN label (#847) * fix(orb-ui): agent policy fixes (#843) * feat(orb-ui): agent policy handler add modal * wip - agent policy adjustments * fix(orb-ui): qname validator and other fixes * fix(orb-ui): #487 last modified field on policies list page (#844) * fix last modified date on agent policy list * agent subscription to multiple groups (orb-tags) and tag scenarios * agent subscription to multiple groups (orb-tags) and tag scenarios * fix(orb-ui): fix regex pattern for fullname in register form (#845) * Bug/orb mix tags (#848) * trying approach with group tags * merge tag columns to fix bug on matching tags * fix(orb-ui): fix agent policy edit failing (#855) * fix(orb-ui): Add cancel button to all stepper controls in forms (#846) * fix(orb-ui): Add cancel button to all stepper controls in forms * Agent Create/Edit * Agent Group/Edit * Agent Policy Create/Edit * Dataset Create/Edit * Sink Create/Edit * fix(orb-ui): fix strings standardization and codestyle * add separate orb-agent workflow * add push step to new orb-agent workflow * fix(orb-ui): fix validator for having at least one module in agent policy (#858) * Tests/integration tests (#871) * orb agent tags editing * Feature/orb fleet increase unit tests coverage (#801) * test(fleet): improvement on unit tests coverage of data access layer * test(fleet): improvements on agent service unit tests coverage * test(fleet): improvements on api unit tests coverage * test(fleet): increase unit tests coverage of fleet service * test(fleet): first unit test of comms service * test(fleet): improvements on service and endpoint unit tests and minor corrections on endpoint response * Test(fleet): unit tests for comms service * test(fleet): add a mock nats PubSub * edit agent name and tag scenarios (#872) * Test/orb fleet unit test mix tags (#868) * create scenarios to test mix of agent and orb tags * create more test cases for matching agents * new test case for unmatching tags * fix expected return of matching agents feature on postgres's unit test (#874) * fix broken unit tests on fleet agents (#877) * Tests/editing group tags name description (#876) * editing groups' tags, name and description Co-authored-by: iatdaitan <86745196+iatdaitan@users.noreply.github.com> Co-authored-by: mclcavalcante <mcavalcante@daitan.com> Co-authored-by: manrodrigues <78241475+manrodrigues@users.noreply.github.com> Co-authored-by: Stéfano de Faria <sfaria@daitan.com> Co-authored-by: gpazuch <1490938+gpazuch@users.noreply.github.com> Co-authored-by: dscabral <dcabral@daitan.com> Co-authored-by: Stéfano <93345073+stefano-daitan@users.noreply.github.com> Co-authored-by: dscabral <87036053+dscabral@users.noreply.github.com> Co-authored-by: Shannon Weyrick <sweyrick@ns1.com>
- Loading branch information