Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Python] push_prompt update #1202

Merged
merged 2 commits into from
Nov 11, 2024
Merged

[Python] push_prompt update #1202

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
940032d
[Python] push_prompt update
hinthornw Nov 11, 2024
20b17dd
Fix check for update
hinthornw Nov 11, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 7 additions & 5 deletions python/langsmith/client.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
"""Client for interacting with the LangSmith API.

Check notice on line 1 in python/langsmith/client.py

View workflow job for this annotation

GitHub Actions / benchmark

Benchmark results 

.........................................
create_5_000_run_trees: Mean +- std dev: 608 ms +- 44 ms
.........................................
create_10_000_run_trees: Mean +- std dev: 1.18 sec +- 0.05 sec
.........................................
create_20_000_run_trees: Mean +- std dev: 1.18 sec +- 0.05 sec
.........................................
dumps_class_nested_py_branch_and_leaf_200x400: Mean +- std dev: 709 us +- 12 us
.........................................
dumps_class_nested_py_leaf_50x100: Mean +- std dev: 25.0 ms +- 0.4 ms
.........................................
dumps_class_nested_py_leaf_100x200: Mean +- std dev: 103 ms +- 2 ms
.........................................
dumps_dataclass_nested_50x100: Mean +- std dev: 25.2 ms +- 0.2 ms
.........................................
WARNING: the benchmark result may be unstable
* the standard deviation (15.5 ms) is 24% of the mean (65.6 ms)
* the maximum (102 ms) is 55% greater than the mean (65.6 ms)

Try to rerun the benchmark with more runs, values and/or loops.
Run 'python -m pyperf system tune' command to reduce the system jitter.
Use pyperf stats, pyperf dump and pyperf hist to analyze results.
Use --quiet option to hide these warnings.

dumps_pydantic_nested_50x100: Mean +- std dev: 65.6 ms +- 15.5 ms
.........................................
WARNING: the benchmark result may be unstable
* the standard deviation (29.2 ms) is 14% of the mean (216 ms)

Try to rerun the benchmark with more runs, values and/or loops.
Run 'python -m pyperf system tune' command to reduce the system jitter.
Use pyperf stats, pyperf dump and pyperf hist to analyze results.
Use --quiet option to hide these warnings.

dumps_pydanticv1_nested_50x100: Mean +- std dev: 216 ms +- 29 ms

Check notice on line 1 in python/langsmith/client.py

View workflow job for this annotation

GitHub Actions / benchmark

Comparison against main 

+------------------------------------+----------+------------------------+
| Benchmark                          | main     | changes                |
+====================================+==========+========================+
| create_5_000_run_trees             | 621 ms   | 608 ms: 1.02x faster   |
+------------------------------------+----------+------------------------+
| create_10_000_run_trees            | 1.20 sec | 1.18 sec: 1.02x faster |
+------------------------------------+----------+------------------------+
| dumps_class_nested_py_leaf_100x200 | 104 ms   | 103 ms: 1.01x faster   |
+------------------------------------+----------+------------------------+
| create_20_000_run_trees            | 1.19 sec | 1.18 sec: 1.01x faster |
+------------------------------------+----------+------------------------+
| dumps_dataclass_nested_50x100      | 25.4 ms  | 25.2 ms: 1.01x faster  |
+------------------------------------+----------+------------------------+
| dumps_class_nested_py_leaf_50x100  | 25.1 ms  | 25.0 ms: 1.00x faster  |
+------------------------------------+----------+------------------------+
| Geometric mean                     | (ref)    | 1.01x faster           |
+------------------------------------+----------+------------------------+

Benchmark hidden because not significant (3): dumps_pydanticv1_nested_50x100, dumps_pydantic_nested_50x100, dumps_class_nested_py_branch_and_leaf_200x400

Use the client to customize API keys / workspace ocnnections, SSl certs,
etc. for tracing.
Expand Down Expand Up @@ -5611,7 +5611,7 @@
*,
object: Optional[Any] = None,
parent_commit_hash: str = "latest",
is_public: bool = False,
is_public: Optional[bool] = None,
description: Optional[str] = None,
readme: Optional[str] = None,
tags: Optional[Sequence[str]] = None,
Expand All @@ -5628,7 +5628,10 @@
object (Optional[Any]): The LangChain object to push.
parent_commit_hash (str): The parent commit hash.
Defaults to "latest".
is_public (bool): Whether the prompt should be public. Defaults to False.
is_public (Optional[bool]): Whether the prompt should be public.
If None (default), the current visibility status is maintained for existing prompts.
For new prompts, None defaults to private.
Set to True to make public, or False to make private.
description (Optional[str]): A description of the prompt.
Defaults to an empty string.
readme (Optional[str]): A readme for the prompt.
Expand All @@ -5643,8 +5646,7 @@
# Create or update prompt metadata
if self._prompt_exists(prompt_identifier):
if any(
param is not None
for param in [parent_commit_hash, is_public, description, readme, tags]
param is not None for param in [is_public, description, readme, tags]
):
self.update_prompt(
prompt_identifier,
Expand All @@ -5656,7 +5658,7 @@
else:
self.create_prompt(
prompt_identifier,
is_public=is_public,
is_public=is_public if is_public is not None else False,
description=description,
readme=readme,
tags=tags,
Expand Down
6 changes: 3 additions & 3 deletions python/tests/unit_tests/test_run_helpers.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -372,9 +372,9 @@ def my_stream_fn(a, b, d, **kwargs):
]
first_patch = next((d for d in call_data if d.get("patch")), None)
attempt += 1

assert first_patch["name"] == "my_stream_fn"
assert first_patch[0]["outputs"] == {"my_output": expected}
if "name" in first_patch:
assert first_patch["name"] == "my_stream_fn"
assert first_patch[0]["outputs"] == {"my_output": expected}


@pytest.mark.parametrize("use_next", [True, False])
Expand Down
Loading