OpenAI and Bedrock Preview (#971)

* Add OpenAI Test Infrastructure (#926) * Add openai to tox * Add OpenAI test files. * Add test functions. * [Mega-Linter] Apply linters fixes --------- Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: mergify[bot] <mergify[bot]@users.noreply.github.com> * OpenAI Mock Backend (#929) * Add mock external openai server * Add mocked OpenAI server fixtures * Set up recorded responses. * Clean mock server to depend on http server * Linting * Pin flask version for flask restx tests. (#931) * Ignore new redis methods. (#932) Co-authored-by: Lalleh Rafeei <84813886+lrafeei@users.noreply.github.com> * Remove approved paths * Update CI Image (#930) * Update available python versions in CI * Update makefile with overrides * Fix default branch detection for arm builds --------- Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * Add mocking for embedding endpoint * [Mega-Linter] Apply linters fixes * Add ratelimit headers * [Mega-Linter] Apply linters fixes * Only get package version once (#928) * Only get package version once * Add disconnect method * Add disconnect method --------- Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * Add datalib dependency for embedding testing. * Add OpenAI Test Infrastructure (#926) * Add openai to tox * Add OpenAI test files. * Add test functions. * [Mega-Linter] Apply linters fixes --------- Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: mergify[bot] <mergify[bot]@users.noreply.github.com> * Add mock external openai server * Add mocked OpenAI server fixtures * Set up recorded responses. * Clean mock server to depend on http server * Linting * Remove approved paths * Add mocking for embedding endpoint * [Mega-Linter] Apply linters fixes * Add ratelimit headers * [Mega-Linter] Apply linters fixes * Add datalib dependency for embedding testing. --------- Co-authored-by: Uma Annamalai <uannamalai@newrelic.com> Co-authored-by: Lalleh Rafeei <84813886+lrafeei@users.noreply.github.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: TimPansino <TimPansino@users.noreply.github.com> Co-authored-by: Hannah Stepanek <hstepanek@newrelic.com> Co-authored-by: mergify[bot] <mergify[bot]@users.noreply.github.com> * Update OpenAI testing infra to match bedrock (#939) * Add OpenAI sync chat completion instrumentation (#934) * Add openai sync instrumentation. * Remove commented code. * Test cleanup. * Add request/ response IDs. * Fixups. * Add conversation ID to message events. --------- Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * Add OpenAI sync embedding instrumentation (#938) * Add sync instrumentation for OpenAI embeddings. * Remove comments. * Clean up embedding event dictionary. * Update response_time to duration. * Linting fixes. * [Mega-Linter] Apply linters fixes * Trigger tests --------- Co-authored-by: umaannamalai <umaannamalai@users.noreply.github.com> Co-authored-by: Hannah Stepanek <hstepanek@newrelic.com> * Instrument acreate's for open-ai (#935) * Instrument acreate's for open ai async * Remove duplicated vendor * Re-use expected & input payloads in tests * Attach ml_event to APM entity by default (#940) * Attach non InferenceEvents to APM entity * Validate both resource payloads * Add tests for non-inference events * Add OpenAI sync embedding instrumentation (#938) * Add sync instrumentation for OpenAI embeddings. * Remove comments. * Clean up embedding event dictionary. * Update response_time to duration. * Linting fixes. * [Mega-Linter] Apply linters fixes * Trigger tests --------- Co-authored-by: umaannamalai <umaannamalai@users.noreply.github.com> Co-authored-by: Hannah Stepanek <hstepanek@newrelic.com> * Fixup: test names --------- Co-authored-by: Uma Annamalai <uannamalai@newrelic.com> Co-authored-by: umaannamalai <umaannamalai@users.noreply.github.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * Add truncation for ML events. (#943) * Add 4096 char truncation for ML events. * Add max attr check. * Fixup. * Fix character length ml event test. * Ignore test_ml_events.py for Py2. * Cleanup custom event if checks. * Add import statement. --------- Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * Add framework metric for OpenAI. (#945) * Add framework metric for OpenAI. * [Mega-Linter] Apply linters fixes * Trigger tests * Fix missing version info. * [Mega-Linter] Apply linters fixes --------- Co-authored-by: umaannamalai <umaannamalai@users.noreply.github.com> Co-authored-by: Hannah Stepanek <hstepanek@newrelic.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * Add truncation support for ML events recorded outside txns. (#949) * Add ml tests for outside transaction. * Update validator. * Add ML flag to application code path for record_ml_event. * Bedrock Testing Infrastructure (#937) * Add AWS Bedrock testing infrastructure * Cache Package Version Lookups (#946) * Cache _get_package_version * Add Python 2.7 support to get_package_version caching * [Mega-Linter] Apply linters fixes * Bump tests --------- Co-authored-by: SlavaSkvortsov <29122694+SlavaSkvortsov@users.noreply.github.com> Co-authored-by: TimPansino <TimPansino@users.noreply.github.com> * Fix Redis Generator Methods (#947) * Fix scan_iter for redis * Replace generator methods * Update instance info instrumentation * Remove mistake from uninstrumented methods * Add skip condition to asyncio generator tests * Add skip condition to asyncio generator tests --------- Co-authored-by: Lalleh Rafeei <lrafeei@newrelic.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * Automatic RPM System Updates (#948) * Checkout old action * Adding RPM action * Add dry run * Incorporating action into workflow * Wire secret into custom action * Enable action * Correct action name * Fix syntax * Fix quoting issues * Drop pre-verification. Does not work on python * Fix merge artifact * Remove OpenAI references --------- Co-authored-by: Uma Annamalai <uannamalai@newrelic.com> Co-authored-by: SlavaSkvortsov <29122694+SlavaSkvortsov@users.noreply.github.com> Co-authored-by: TimPansino <TimPansino@users.noreply.github.com> Co-authored-by: Lalleh Rafeei <lrafeei@newrelic.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * Mock openai error responses (#950) * Add example tests and mock error responses * Set invalid api key in auth error test Co-authored-by: Timothy Pansino <11214426+TimPansino@users.noreply.github.com> --------- Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: Timothy Pansino <11214426+TimPansino@users.noreply.github.com> * OpenAI ErrorTrace attributes (#941) * Add openai sync instrumentation. * Remove commented code. * Initial openai error commit * Add example tests and mock error responses * Changes to attribute collection * Change error tests to match mock server * [Mega-Linter] Apply linters fixes * Trigger tests * Add dt_enabled decorator to error tests * Add embedded and async error tests * [Mega-Linter] Apply linters fixes * Trigger tests * Add http.statusCode to span before notice_error call * Report number of messages in error trace even if 0 * Revert notice_error and add _nr_message attr * Remove enabled_ml_settings as not needed * Add stats engine _nr_message test * [Mega-Linter] Apply linters fixes * Trigger tests * Revert black formatting in unicode/byte messages --------- Co-authored-by: Uma Annamalai <uannamalai@newrelic.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: Hannah Stepanek <hstepanek@newrelic.com> Co-authored-by: lrafeei <lrafeei@users.noreply.github.com> Co-authored-by: hmstepanek <hmstepanek@users.noreply.github.com> * Bedrock Sync Chat Completion Instrumentation (#953) * Add AWS Bedrock testing infrastructure * Squashed commit of the following: commit 2834663 Author: Timothy Pansino <11214426+TimPansino@users.noreply.github.com> Date: Mon Oct 9 17:42:05 2023 -0700 OpenAI Mock Backend (#929) * Add mock external openai server * Add mocked OpenAI server fixtures * Set up recorded responses. * Clean mock server to depend on http server * Linting * Pin flask version for flask restx tests. (#931) * Ignore new redis methods. (#932) Co-authored-by: Lalleh Rafeei <84813886+lrafeei@users.noreply.github.com> * Remove approved paths * Update CI Image (#930) * Update available python versions in CI * Update makefile with overrides * Fix default branch detection for arm builds --------- Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * Add mocking for embedding endpoint * [Mega-Linter] Apply linters fixes * Add ratelimit headers * [Mega-Linter] Apply linters fixes * Only get package version once (#928) * Only get package version once * Add disconnect method * Add disconnect method --------- Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * Add datalib dependency for embedding testing. * Add OpenAI Test Infrastructure (#926) * Add openai to tox * Add OpenAI test files. * Add test functions. * [Mega-Linter] Apply linters fixes --------- Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: mergify[bot] <mergify[bot]@users.noreply.github.com> * Add mock external openai server * Add mocked OpenAI server fixtures * Set up recorded responses. * Clean mock server to depend on http server * Linting * Remove approved paths * Add mocking for embedding endpoint * [Mega-Linter] Apply linters fixes * Add ratelimit headers * [Mega-Linter] Apply linters fixes * Add datalib dependency for embedding testing. --------- Co-authored-by: Uma Annamalai <uannamalai@newrelic.com> Co-authored-by: Lalleh Rafeei <84813886+lrafeei@users.noreply.github.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: TimPansino <TimPansino@users.noreply.github.com> Co-authored-by: Hannah Stepanek <hstepanek@newrelic.com> Co-authored-by: mergify[bot] <mergify[bot]@users.noreply.github.com> commit db63d45 Author: Uma Annamalai <uannamalai@newrelic.com> Date: Mon Oct 2 15:31:38 2023 -0700 Add OpenAI Test Infrastructure (#926) * Add openai to tox * Add OpenAI test files. * Add test functions. * [Mega-Linter] Apply linters fixes --------- Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: mergify[bot] <mergify[bot]@users.noreply.github.com> * Squashed commit of the following: commit 182c7a8 Author: Uma Annamalai <uannamalai@newrelic.com> Date: Fri Oct 13 10:12:55 2023 -0700 Add request/ response IDs. commit f6d13f8 Author: Uma Annamalai <uannamalai@newrelic.com> Date: Thu Oct 12 13:23:39 2023 -0700 Test cleanup. commit d057663 Author: Uma Annamalai <uannamalai@newrelic.com> Date: Tue Oct 10 10:23:00 2023 -0700 Remove commented code. commit dd29433 Author: Uma Annamalai <uannamalai@newrelic.com> Date: Tue Oct 10 10:19:01 2023 -0700 Add openai sync instrumentation. commit 2834663 Author: Timothy Pansino <11214426+TimPansino@users.noreply.github.com> Date: Mon Oct 9 17:42:05 2023 -0700 OpenAI Mock Backend (#929) * Add mock external openai server * Add mocked OpenAI server fixtures * Set up recorded responses. * Clean mock server to depend on http server * Linting * Pin flask version for flask restx tests. (#931) * Ignore new redis methods. (#932) Co-authored-by: Lalleh Rafeei <84813886+lrafeei@users.noreply.github.com> * Remove approved paths * Update CI Image (#930) * Update available python versions in CI * Update makefile with overrides * Fix default branch detection for arm builds --------- Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * Add mocking for embedding endpoint * [Mega-Linter] Apply linters fixes * Add ratelimit headers * [Mega-Linter] Apply linters fixes * Only get package version once (#928) * Only get package version once * Add disconnect method * Add disconnect method --------- Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * Add datalib dependency for embedding testing. * Add OpenAI Test Infrastructure (#926) * Add openai to tox * Add OpenAI test files. * Add test functions. * [Mega-Linter] Apply linters fixes --------- Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: mergify[bot] <mergify[bot]@users.noreply.github.com> * Add mock external openai server * Add mocked OpenAI server fixtures * Set up recorded responses. * Clean mock server to depend on http server * Linting * Remove approved paths * Add mocking for embedding endpoint * [Mega-Linter] Apply linters fixes * Add ratelimit headers * [Mega-Linter] Apply linters fixes * Add datalib dependency for embedding testing. --------- Co-authored-by: Uma Annamalai <uannamalai@newrelic.com> Co-authored-by: Lalleh Rafeei <84813886+lrafeei@users.noreply.github.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: TimPansino <TimPansino@users.noreply.github.com> Co-authored-by: Hannah Stepanek <hstepanek@newrelic.com> Co-authored-by: mergify[bot] <mergify[bot]@users.noreply.github.com> commit db63d45 Author: Uma Annamalai <uannamalai@newrelic.com> Date: Mon Oct 2 15:31:38 2023 -0700 Add OpenAI Test Infrastructure (#926) * Add openai to tox * Add OpenAI test files. * Add test functions. * [Mega-Linter] Apply linters fixes --------- Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: mergify[bot] <mergify[bot]@users.noreply.github.com> * Cache Package Version Lookups (#946) * Cache _get_package_version * Add Python 2.7 support to get_package_version caching * [Mega-Linter] Apply linters fixes * Bump tests --------- Co-authored-by: SlavaSkvortsov <29122694+SlavaSkvortsov@users.noreply.github.com> Co-authored-by: TimPansino <TimPansino@users.noreply.github.com> * Fix Redis Generator Methods (#947) * Fix scan_iter for redis * Replace generator methods * Update instance info instrumentation * Remove mistake from uninstrumented methods * Add skip condition to asyncio generator tests * Add skip condition to asyncio generator tests --------- Co-authored-by: Lalleh Rafeei <lrafeei@newrelic.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * TEMP * Automatic RPM System Updates (#948) * Checkout old action * Adding RPM action * Add dry run * Incorporating action into workflow * Wire secret into custom action * Enable action * Correct action name * Fix syntax * Fix quoting issues * Drop pre-verification. Does not work on python * Fix merge artifact * Bedrock titan extraction nearly complete * Cleaning up titan bedrock implementation * TEMP * Tests for bedrock passing Co-authored-by: Lalleh Rafeei <lrafeei@users.noreply.github.com> * Cleaned up titan testing Co-authored-by: Lalleh Rafeei <lrafeei@users.noreply.github.com> Co-authored-by: Hannah Stepanek <hmstepanek@users.noreply.github.com> * Parametrized bedrock testing * Add support for AI21-J2 models * Change to dynamic no conversation id events * Drop all openai refs * [Mega-Linter] Apply linters fixes * Adding response_id and response_model * Drop python 3.7 tests for Hypercorn (#954) * Apply suggestions from code review * Remove unused import --------- Co-authored-by: Uma Annamalai <uannamalai@newrelic.com> Co-authored-by: SlavaSkvortsov <29122694+SlavaSkvortsov@users.noreply.github.com> Co-authored-by: TimPansino <TimPansino@users.noreply.github.com> Co-authored-by: Lalleh Rafeei <lrafeei@newrelic.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: Lalleh Rafeei <lrafeei@users.noreply.github.com> Co-authored-by: Hannah Stepanek <hmstepanek@users.noreply.github.com> Co-authored-by: Lalleh Rafeei <84813886+lrafeei@users.noreply.github.com> * Feature bedrock cohere instrumentation (#955) * Add AWS Bedrock testing infrastructure * Squashed commit of the following: commit 2834663 Author: Timothy Pansino <11214426+TimPansino@users.noreply.github.com> Date: Mon Oct 9 17:42:05 2023 -0700 OpenAI Mock Backend (#929) * Add mock external openai server * Add mocked OpenAI server fixtures * Set up recorded responses. * Clean mock server to depend on http server * Linting * Pin flask version for flask restx tests. (#931) * Ignore new redis methods. (#932) Co-authored-by: Lalleh Rafeei <84813886+lrafeei@users.noreply.github.com> * Remove approved paths * Update CI Image (#930) * Update available python versions in CI * Update makefile with overrides * Fix default branch detection for arm builds --------- Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * Add mocking for embedding endpoint * [Mega-Linter] Apply linters fixes * Add ratelimit headers * [Mega-Linter] Apply linters fixes * Only get package version once (#928) * Only get package version once * Add disconnect method * Add disconnect method --------- Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * Add datalib dependency for embedding testing. * Add OpenAI Test Infrastructure (#926) * Add openai to tox * Add OpenAI test files. * Add test functions. * [Mega-Linter] Apply linters fixes --------- Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: mergify[bot] <mergify[bot]@users.noreply.github.com> * Add mock external openai server * Add mocked OpenAI server fixtures * Set up recorded responses. * Clean mock server to depend on http server * Linting * Remove approved paths * Add mocking for embedding endpoint * [Mega-Linter] Apply linters fixes * Add ratelimit headers * [Mega-Linter] Apply linters fixes * Add datalib dependency for embedding testing. --------- Co-authored-by: Uma Annamalai <uannamalai@newrelic.com> Co-authored-by: Lalleh Rafeei <84813886+lrafeei@users.noreply.github.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: TimPansino <TimPansino@users.noreply.github.com> Co-authored-by: Hannah Stepanek <hstepanek@newrelic.com> Co-authored-by: mergify[bot] <mergify[bot]@users.noreply.github.com> commit db63d45 Author: Uma Annamalai <uannamalai@newrelic.com> Date: Mon Oct 2 15:31:38 2023 -0700 Add OpenAI Test Infrastructure (#926) * Add openai to tox * Add OpenAI test files. * Add test functions. * [Mega-Linter] Apply linters fixes --------- Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: mergify[bot] <mergify[bot]@users.noreply.github.com> * Squashed commit of the following: commit 182c7a8 Author: Uma Annamalai <uannamalai@newrelic.com> Date: Fri Oct 13 10:12:55 2023 -0700 Add request/ response IDs. commit f6d13f8 Author: Uma Annamalai <uannamalai@newrelic.com> Date: Thu Oct 12 13:23:39 2023 -0700 Test cleanup. commit d057663 Author: Uma Annamalai <uannamalai@newrelic.com> Date: Tue Oct 10 10:23:00 2023 -0700 Remove commented code. commit dd29433 Author: Uma Annamalai <uannamalai@newrelic.com> Date: Tue Oct 10 10:19:01 2023 -0700 Add openai sync instrumentation. commit 2834663 Author: Timothy Pansino <11214426+TimPansino@users.noreply.github.com> Date: Mon Oct 9 17:42:05 2023 -0700 OpenAI Mock Backend (#929) * Add mock external openai server * Add mocked OpenAI server fixtures * Set up recorded responses. * Clean mock server to depend on http server * Linting * Pin flask version for flask restx tests. (#931) * Ignore new redis methods. (#932) Co-authored-by: Lalleh Rafeei <84813886+lrafeei@users.noreply.github.com> * Remove approved paths * Update CI Image (#930) * Update available python versions in CI * Update makefile with overrides * Fix default branch detection for arm builds --------- Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * Add mocking for embedding endpoint * [Mega-Linter] Apply linters fixes * Add ratelimit headers * [Mega-Linter] Apply linters fixes * Only get package version once (#928) * Only get package version once * Add disconnect method * Add disconnect method --------- Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * Add datalib dependency for embedding testing. * Add OpenAI Test Infrastructure (#926) * Add openai to tox * Add OpenAI test files. * Add test functions. * [Mega-Linter] Apply linters fixes --------- Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: mergify[bot] <mergify[bot]@users.noreply.github.com> * Add mock external openai server * Add mocked OpenAI server fixtures * Set up recorded responses. * Clean mock server to depend on http server * Linting * Remove approved paths * Add mocking for embedding endpoint * [Mega-Linter] Apply linters fixes * Add ratelimit headers * [Mega-Linter] Apply linters fixes * Add datalib dependency for embedding testing. --------- Co-authored-by: Uma Annamalai <uannamalai@newrelic.com> Co-authored-by: Lalleh Rafeei <84813886+lrafeei@users.noreply.github.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: TimPansino <TimPansino@users.noreply.github.com> Co-authored-by: Hannah Stepanek <hstepanek@newrelic.com> Co-authored-by: mergify[bot] <mergify[bot]@users.noreply.github.com> commit db63d45 Author: Uma Annamalai <uannamalai@newrelic.com> Date: Mon Oct 2 15:31:38 2023 -0700 Add OpenAI Test Infrastructure (#926) * Add openai to tox * Add OpenAI test files. * Add test functions. * [Mega-Linter] Apply linters fixes --------- Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: mergify[bot] <mergify[bot]@users.noreply.github.com> * TEMP * Bedrock titan extraction nearly complete * Cleaning up titan bedrock implementation * TEMP * Tests for bedrock passing Co-authored-by: Lalleh Rafeei <lrafeei@users.noreply.github.com> * Cleaned up titan testing Co-authored-by: Lalleh Rafeei <lrafeei@users.noreply.github.com> Co-authored-by: Hannah Stepanek <hmstepanek@users.noreply.github.com> * Parametrized bedrock testing * Add support for AI21-J2 models * Change to dynamic no conversation id events * Add cohere model * Remove openai instrumentation from this branch * Remove OpenAI from newrelic/config.py --------- Co-authored-by: Uma Annamalai <uannamalai@newrelic.com> Co-authored-by: Tim Pansino <timpansino@gmail.com> Co-authored-by: Lalleh Rafeei <lrafeei@users.noreply.github.com> Co-authored-by: Hannah Stepanek <hmstepanek@users.noreply.github.com> * AWS Bedrock Embedding Instrumentation (#957) * AWS Bedrock embedding instrumentation * Correct symbol name * Add support for bedrock claude (#960) Co-authored-by: Timothy Pansino <11214426+TimPansino@users.noreply.github.com> * Combine Botocore Tests (#959) * Initial file migration * Enable DT on all span tests * Add pytest skip for older botocore versions * Fixup: app name merge conflict --------- Co-authored-by: Hannah Stepanek <hstepanek@newrelic.com> * Pin openai tests to below 1.0 (#962) * Pin openai below 1.0 * Fixup * Add openai feedback support (#942) * Add get_ai_message_ids & message id capturing * Add tests * Remove generator * Add tests for conversation id unset * Add error code to mocked responses * Remove bedrock tests --------- Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: Uma Annamalai <uannamalai@newrelic.com> * Add ingest source to openai events (#961) * Pin openai below 1.0 * Fixup * Add ingest_source to events * Remove duplicate test file * Handle 0.32.0.post1 version in tests (#963) --------- Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * Handle 0.32.0.post1 version in tests (#963) * Initial merge commit * Update moto * Test for Bedrock embeddings metrics * Add record_llm_feedback_event API (#964) * Implement record_ai_feedback API. * [Mega-Linter] Apply linters fixes * Change API name to record_ai_feedback_event. * Fix API naming. * Rename to record_llm_feedback_event and get_llm_message_ids. * [Mega-Linter] Apply linters fixes * Address review feedback. * Update test structure. * [Mega-Linter] Apply linters fixes * Bump tests. --------- Co-authored-by: umaannamalai <umaannamalai@users.noreply.github.com> * Bedrock Error Tracing (#966) * Cache Package Version Lookups (#946) * Cache _get_package_version * Add Python 2.7 support to get_package_version caching * [Mega-Linter] Apply linters fixes * Bump tests --------- Co-authored-by: SlavaSkvortsov <29122694+SlavaSkvortsov@users.noreply.github.com> Co-authored-by: TimPansino <TimPansino@users.noreply.github.com> * Fix Redis Generator Methods (#947) * Fix scan_iter for redis * Replace generator methods * Update instance info instrumentation * Remove mistake from uninstrumented methods * Add skip condition to asyncio generator tests * Add skip condition to asyncio generator tests --------- Co-authored-by: Lalleh Rafeei <lrafeei@newrelic.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * Automatic RPM System Updates (#948) * Checkout old action * Adding RPM action * Add dry run * Incorporating action into workflow * Wire secret into custom action * Enable action * Correct action name * Fix syntax * Fix quoting issues * Drop pre-verification. Does not work on python * Fix merge artifact * Drop python 3.7 tests for Hypercorn (#954) * Fix pyenv installation for devcontainer (#936) Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * Remove duplicate kafka import hook (#956) Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * Initial bedrock error tracing commit * Handle 0.32.0.post1 version in tests (#963) * Add status code to mock bedrock server * Updating error response recording logic * Work on bedrock errror tracing * Chat completion error tracing * Adding embedding error tracing * Delete comment * Update moto --------- Co-authored-by: SlavaSkvortsov <29122694+SlavaSkvortsov@users.noreply.github.com> Co-authored-by: TimPansino <TimPansino@users.noreply.github.com> Co-authored-by: Lalleh Rafeei <lrafeei@newrelic.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: Lalleh Rafeei <84813886+lrafeei@users.noreply.github.com> Co-authored-by: Hannah Stepanek <hstepanek@newrelic.com> * Fix expected chat completion tests * Remove commented out code * Correct Bedrock metric name --------- Co-authored-by: Uma Annamalai <uannamalai@newrelic.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: mergify[bot] <mergify[bot]@users.noreply.github.com> Co-authored-by: Timothy Pansino <11214426+TimPansino@users.noreply.github.com> Co-authored-by: TimPansino <TimPansino@users.noreply.github.com> Co-authored-by: Hannah Stepanek <hstepanek@newrelic.com> Co-authored-by: umaannamalai <umaannamalai@users.noreply.github.com> Co-authored-by: SlavaSkvortsov <29122694+SlavaSkvortsov@users.noreply.github.com> Co-authored-by: lrafeei <lrafeei@users.noreply.github.com> Co-authored-by: hmstepanek <hmstepanek@users.noreply.github.com> Co-authored-by: Tim Pansino <timpansino@gmail.com>
newrelic · Nov 14, 2023 · 498126a · 498126a
1 parent b2e9e74
commit 498126a
Show file tree

Hide file tree

Showing 40 changed files with 7,612 additions and 202 deletions.
diff --git a/newrelic/agent.py b/newrelic/agent.py
@@ -153,6 +153,10 @@ def __asgi_application(*args, **kwargs):
 from newrelic.api.message_transaction import (
     wrap_message_transaction as __wrap_message_transaction,
 )
+from newrelic.api.ml_model import get_llm_message_ids as __get_llm_message_ids
+from newrelic.api.ml_model import (
+    record_llm_feedback_event as __record_llm_feedback_event,
+)
 from newrelic.api.ml_model import wrap_mlmodel as __wrap_mlmodel
 from newrelic.api.profile_trace import ProfileTraceWrapper as __ProfileTraceWrapper
 from newrelic.api.profile_trace import profile_trace as __profile_trace
@@ -340,3 +344,5 @@ def __asgi_application(*args, **kwargs):
 insert_html_snippet = __wrap_api_call(__insert_html_snippet, "insert_html_snippet")
 verify_body_exists = __wrap_api_call(__verify_body_exists, "verify_body_exists")
 wrap_mlmodel = __wrap_api_call(__wrap_mlmodel, "wrap_mlmodel")
+get_llm_message_ids = __wrap_api_call(__get_llm_message_ids, "get_llm_message_ids")
+record_llm_feedback_event = __wrap_api_call(__record_llm_feedback_event, "record_llm_feedback_event")
diff --git a/newrelic/api/ml_model.py b/newrelic/api/ml_model.py
@@ -13,7 +13,10 @@
 # limitations under the License.
 
 import sys
+import uuid
+import warnings
 
+from newrelic.api.transaction import current_transaction
 from newrelic.common.object_names import callable_name
 from newrelic.hooks.mlmodel_sklearn import _nr_instrument_model
 
@@ -33,3 +36,49 @@ def wrap_mlmodel(model, name=None, version=None, feature_names=None, label_names
         model._nr_wrapped_label_names = label_names
     if metadata:
         model._nr_wrapped_metadata = metadata
+
+
+def get_llm_message_ids(response_id=None):
+    transaction = current_transaction()
+    if response_id and transaction:
+        nr_message_ids = getattr(transaction, "_nr_message_ids", {})
+        message_id_info = nr_message_ids.pop(response_id, ())
+
+        if not message_id_info:
+            warnings.warn("No message ids found for %s" % response_id)
+            return []
+
+        conversation_id, request_id, ids = message_id_info
+
+        return [{"conversation_id": conversation_id, "request_id": request_id, "message_id": _id} for _id in ids]
+    warnings.warn("No message ids found. get_llm_message_ids must be called within the scope of a transaction.")
+    return []
+
+
+def record_llm_feedback_event(
+    message_id, rating, conversation_id=None, request_id=None, category=None, message=None, metadata=None
+):
+    transaction = current_transaction()
+    if not transaction:
+        warnings.warn(
+            "No message feedback events will be recorded. record_llm_feedback_event must be called within the "
+            "scope of a transaction."
+        )
+        return
+
+    feedback_message_id = str(uuid.uuid4())
+    metadata = metadata or {}
+
+    feedback_message_event = {
+        "id": feedback_message_id,
+        "message_id": message_id,
+        "rating": rating,
+        "conversation_id": conversation_id or "",
+        "request_id": request_id or "",
+        "category": category or "",
+        "message": message or "",
+        "ingest_source": "Python",
+    }
+    feedback_message_event.update(metadata)
+
+    transaction.record_ml_event("LlmFeedbackMessage", feedback_message_event)
diff --git a/newrelic/api/time_trace.py b/newrelic/api/time_trace.py
@@ -29,7 +29,6 @@
 )
 from newrelic.core.config import is_expected_error, should_ignore_error
 from newrelic.core.trace_cache import trace_cache
-
 from newrelic.packages import six
 
 _logger = logging.getLogger(__name__)
@@ -260,6 +259,11 @@ def _observe_exception(self, exc_info=None, ignore=None, expected=None, status_c
         module, name, fullnames, message_raw = parse_exc_info((exc, value, tb))
         fullname = fullnames[0]
 
+        # In case message is in JSON format for OpenAI models
+        # this will result in a "cleaner" message format
+        if getattr(value, "_nr_message", None):
+            message_raw = value._nr_message
+
         # Check to see if we need to strip the message before recording it.
 
         if settings.strip_exception_messages.enabled and fullname not in settings.strip_exception_messages.allowlist:
@@ -422,23 +426,32 @@ def notice_error(self, error=None, attributes=None, expected=None, ignore=None,
                         input_attributes = {}
                         input_attributes.update(transaction._custom_params)
                         input_attributes.update(attributes)
-                        error_group_name_raw = settings.error_collector.error_group_callback(value, {
-                            "traceback": tb,
-                            "error.class": exc,
-                            "error.message": message_raw,
-                            "error.expected": is_expected,
-                            "custom_params": input_attributes,
-                            "transactionName": getattr(transaction, "name", None),
-                            "response.status": getattr(transaction, "_response_code", None),
-                            "request.method": getattr(transaction, "_request_method", None),
-                            "request.uri": getattr(transaction, "_request_uri", None),
-                        })
+                        error_group_name_raw = settings.error_collector.error_group_callback(
+                            value,
+                            {
+                                "traceback": tb,
+                                "error.class": exc,
+                                "error.message": message_raw,
+                                "error.expected": is_expected,
+                                "custom_params": input_attributes,
+                                "transactionName": getattr(transaction, "name", None),
+                                "response.status": getattr(transaction, "_response_code", None),
+                                "request.method": getattr(transaction, "_request_method", None),
+                                "request.uri": getattr(transaction, "_request_uri", None),
+                            },
+                        )
                         if error_group_name_raw:
                             _, error_group_name = process_user_attribute("error.group.name", error_group_name_raw)
                             if error_group_name is None or not isinstance(error_group_name, six.string_types):
-                                raise ValueError("Invalid attribute value for error.group.name. Expected string, got: %s" % repr(error_group_name_raw))
+                                raise ValueError(
+                                    "Invalid attribute value for error.group.name. Expected string, got: %s"
+                                    % repr(error_group_name_raw)
+                                )
                     except Exception:
-                        _logger.error("Encountered error when calling error group callback:\n%s", "".join(traceback.format_exception(*sys.exc_info())))
+                        _logger.error(
+                            "Encountered error when calling error group callback:\n%s",
+                            "".join(traceback.format_exception(*sys.exc_info())),
+                        )
                         error_group_name = None
 
             transaction._create_error_node(
@@ -595,13 +608,11 @@ def update_async_exclusive_time(self, min_child_start_time, exclusive_duration):
     def process_child(self, node, is_async):
         self.children.append(node)
         if is_async:
-
             # record the lowest start time
             self.min_child_start_time = min(self.min_child_start_time, node.start_time)
 
             # if there are no children running, finalize exclusive time
             if self.child_count == len(self.children):
-
                 exclusive_duration = node.end_time - self.min_child_start_time
 
                 self.update_async_exclusive_time(self.min_child_start_time, exclusive_duration)

diff --git a/newrelic/api/transaction.py b/newrelic/api/transaction.py
@@ -191,6 +191,7 @@ def __init__(self, application, enabled=None, source=None):
         self._frameworks = set()
         self._message_brokers = set()
         self._dispatchers = set()
+        self._ml_models = set()
 
         self._frozen_path = None
 
@@ -559,6 +560,10 @@ def __exit__(self, exc, value, tb):
             for dispatcher, version in self._dispatchers:
                 self.record_custom_metric("Python/Dispatcher/%s/%s" % (dispatcher, version), 1)
 
+        if self._ml_models:
+            for ml_model, version in self._ml_models:
+                self.record_custom_metric("Python/ML/%s/%s" % (ml_model, version), 1)
+
         if self._settings.distributed_tracing.enabled:
             # Sampled and priority need to be computed at the end of the
             # transaction when distributed tracing or span events are enabled.
@@ -1648,7 +1653,7 @@ def record_ml_event(self, event_type, params):
         if not settings.ml_insights_events.enabled:
             return
 
-        event = create_custom_event(event_type, params)
+        event = create_custom_event(event_type, params, is_ml_event=True)
         if event:
             self._ml_events.add(event, priority=self.priority)
 
@@ -1755,6 +1760,10 @@ def add_dispatcher_info(self, name, version=None):
         if name:
             self._dispatchers.add((name, version))
 
+    def add_ml_model_info(self, name, version=None):
+        if name:
+            self._ml_models.add((name, version))
+
     def dump(self, file):
         """Dumps details about the transaction to the file object."""
 

diff --git a/newrelic/config.py b/newrelic/config.py
@@ -2037,6 +2037,21 @@ def _process_trace_cache_import_hooks():
 
 
 def _process_module_builtin_defaults():
+    _process_module_definition(
+        "openai.api_resources.embedding",
+        "newrelic.hooks.mlmodel_openai",
+        "instrument_openai_api_resources_embedding",
+    )
+    _process_module_definition(
+        "openai.api_resources.chat_completion",
+        "newrelic.hooks.mlmodel_openai",
+        "instrument_openai_api_resources_chat_completion",
+    )
+    _process_module_definition(
+        "openai.util",
+        "newrelic.hooks.mlmodel_openai",
+        "instrument_openai_util",
+    )
     _process_module_definition(
         "asyncio.base_events",
         "newrelic.hooks.coroutines_asyncio",

diff --git a/newrelic/core/application.py b/newrelic/core/application.py
@@ -932,7 +932,7 @@ def record_ml_event(self, event_type, params):
         if settings is None or not settings.ml_insights_events.enabled:
             return
 
-        event = create_custom_event(event_type, params)
+        event = create_custom_event(event_type, params, is_ml_event=True)
 
         if event:
             with self._stats_custom_lock:

diff --git a/newrelic/core/attribute.py b/newrelic/core/attribute.py
@@ -89,6 +89,8 @@
 
 MAX_NUM_USER_ATTRIBUTES = 128
 MAX_ATTRIBUTE_LENGTH = 255
+MAX_NUM_ML_USER_ATTRIBUTES = 64
+MAX_ML_ATTRIBUTE_LENGTH = 4095
 MAX_64_BIT_INT = 2**63 - 1
 MAX_LOG_MESSAGE_LENGTH = 32768
 

diff --git a/newrelic/core/custom_event.py b/newrelic/core/custom_event.py
@@ -18,7 +18,7 @@
 
 from newrelic.core.attribute import (check_name_is_string, check_name_length,
         process_user_attribute, NameIsNotStringException, NameTooLongException,
-        MAX_NUM_USER_ATTRIBUTES)
+        MAX_NUM_USER_ATTRIBUTES, MAX_ML_ATTRIBUTE_LENGTH, MAX_NUM_ML_USER_ATTRIBUTES, MAX_ATTRIBUTE_LENGTH)
 
 
 _logger = logging.getLogger(__name__)
@@ -72,7 +72,8 @@ def process_event_type(name):
     else:
         return name
 
-def create_custom_event(event_type, params):
+
+def create_custom_event(event_type, params, is_ml_event=False):
     """Creates a valid custom event.
 
     Ensures that the custom event has a valid name, and also checks
@@ -83,6 +84,8 @@ def create_custom_event(event_type, params):
     Args:
         event_type (str): The type (name) of the custom event.
         params (dict): Attributes to add to the event.
+        is_ml_event (bool): Boolean indicating whether create_custom_event was called from
+        record_ml_event for truncation purposes
 
     Returns:
         Custom event (list of 2 dicts), if successful.
@@ -99,12 +102,18 @@ def create_custom_event(event_type, params):
 
     try:
         for k, v in params.items():
-            key, value = process_user_attribute(k, v)
+            if is_ml_event:
+                max_length = MAX_ML_ATTRIBUTE_LENGTH
+                max_num_attrs = MAX_NUM_ML_USER_ATTRIBUTES
+            else:
+                max_length = MAX_ATTRIBUTE_LENGTH
+                max_num_attrs = MAX_NUM_USER_ATTRIBUTES
+            key, value = process_user_attribute(k, v, max_length=max_length)
             if key:
-                if len(attributes) >= MAX_NUM_USER_ATTRIBUTES:
+                if len(attributes) >= max_num_attrs:
                     _logger.debug('Maximum number of attributes already '
-                            'added to event %r. Dropping attribute: %r=%r',
-                            name, key, value)
+                                  'added to event %r. Dropping attribute: %r=%r',
+                                  name, key, value)
                 else:
                     attributes[key] = value
     except Exception:

diff --git a/newrelic/core/otlp_utils.py b/newrelic/core/otlp_utils.py
@@ -21,6 +21,7 @@
 
 import logging
 
+from newrelic.api.time_trace import get_service_linking_metadata
 from newrelic.common.encoding_utils import json_encode
 from newrelic.core.config import global_settings
 from newrelic.core.stats_engine import CountStats, TimeStats
@@ -124,8 +125,11 @@ def create_key_values_from_iterable(iterable):
     )
 
 
-def create_resource(attributes=None):
+def create_resource(attributes=None, attach_apm_entity=True):
     attributes = attributes or {"instrumentation.provider": "newrelic-opentelemetry-python-ml"}
+    if attach_apm_entity:
+        metadata = get_service_linking_metadata()
+        attributes.update(metadata)
     return Resource(attributes=create_key_values_from_iterable(attributes))
 
 
@@ -203,7 +207,7 @@ def stats_to_otlp_metrics(metric_data, start_time, end_time):
 
 
 def encode_metric_data(metric_data, start_time, end_time, resource=None, scope=None):
-    resource = resource or create_resource()
+    resource = resource or create_resource(attach_apm_entity=False)
     return MetricsData(
         resource_metrics=[
             ResourceMetrics(
@@ -220,24 +224,45 @@ def encode_metric_data(metric_data, start_time, end_time, resource=None, scope=N
 
 
 def encode_ml_event_data(custom_event_data, agent_run_id):
-    resource = create_resource()
-    ml_events = []
+    # An InferenceEvent is attached to a separate ML Model entity instead
+    # of the APM entity.
+    ml_inference_events = []
+    ml_apm_events = []
     for event in custom_event_data:
         event_info, event_attrs = event
+        event_type = event_info["type"]
         event_attrs.update(
             {
                 "real_agent_id": agent_run_id,
                 "event.domain": "newrelic.ml_events",
-                "event.name": event_info["type"],
+                "event.name": event_type,
             }
         )
         ml_attrs = create_key_values_from_iterable(event_attrs)
         unix_nano_timestamp = event_info["timestamp"] * 1e6
-        ml_events.append(
-            {
-                "time_unix_nano": int(unix_nano_timestamp),
-                "attributes": ml_attrs,
-            }
+        if event_type == "InferenceEvent":
+            ml_inference_events.append(
+                {
+                    "time_unix_nano": int(unix_nano_timestamp),
+                    "attributes": ml_attrs,
+                }
+            )
+        else:
+            ml_apm_events.append(
+                {
+                    "time_unix_nano": int(unix_nano_timestamp),
+                    "attributes": ml_attrs,
+                }
+            )
+
+    resource_logs = []
+    if ml_inference_events:
+        inference_resource = create_resource(attach_apm_entity=False)
+        resource_logs.append(
+            ResourceLogs(resource=inference_resource, scope_logs=[ScopeLogs(log_records=ml_inference_events)])
         )
+    if ml_apm_events:
+        apm_resource = create_resource()
+        resource_logs.append(ResourceLogs(resource=apm_resource, scope_logs=[ScopeLogs(log_records=ml_apm_events)]))
 
-    return LogsData(resource_logs=[ResourceLogs(resource=resource, scope_logs=[ScopeLogs(log_records=ml_events)])])
+    return LogsData(resource_logs=resource_logs)
diff --git a/newrelic/core/stats_engine.py b/newrelic/core/stats_engine.py
@@ -724,6 +724,11 @@ def notice_error(self, error=None, attributes=None, expected=None, ignore=None,
         module, name, fullnames, message_raw = parse_exc_info(error)
         fullname = fullnames[0]
 
+        # In the case case of JSON formatting for OpenAI models
+        # this will result in a "cleaner" message format
+        if getattr(value, "_nr_message", None):
+            message_raw = value._nr_message
+
         # Check to see if we need to strip the message before recording it.
 
         if settings.strip_exception_messages.enabled and fullname not in settings.strip_exception_messages.allowlist: