Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Publish llama-index plugin #924

Merged
merged 7 commits into from
Oct 16, 2024
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/shiny-emus-change.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"livekit-plugins-llama-index": patch
---

Publish llama-index plugin
13 changes: 7 additions & 6 deletions .github/workflows/build-package.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,13 +12,13 @@ on:
workflow_dispatch:
inputs:
package:
description: 'Name of the package to build'
description: "Name of the package to build"
required: true
default: 'livekit-plugins-browser'
default: "livekit-plugins-browser"
artifact_name:
description: 'Artifact name for the distribution package'
description: "Artifact name for the distribution package"
required: true
default: 'build-artifact'
default: "build-artifact"

jobs:
build_plugins:
Expand All @@ -35,7 +35,8 @@ jobs:
inputs.package == 'livekit-plugins-openai' ||
inputs.package == 'livekit-plugins-rag' ||
inputs.package == 'livekit-plugins-silero' ||
inputs.package == 'livekit-plugins-anthropic'
inputs.package == 'livekit-plugins-anthropic' ||
inputs.package == 'livekit-plugins-llama-index'

defaults:
run:
Expand Down Expand Up @@ -98,4 +99,4 @@ jobs:
uses: actions/upload-artifact@v3
with:
name: ${{ inputs.artifact_name }}
path: livekit-plugins/livekit-plugins-browser/dist/
path: livekit-plugins/livekit-plugins-browser/dist/
4 changes: 2 additions & 2 deletions .github/workflows/check-types.yml
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,6 @@ jobs:
with:
submodules: recursive


- uses: actions/setup-python@v5
with:
python-version: "3.9"
Expand All @@ -41,7 +40,8 @@ jobs:
./livekit-plugins/livekit-plugins-cartesia \
./livekit-plugins/livekit-plugins-rag \
./livekit-plugins/livekit-plugins-azure \
./livekit-plugins/livekit-plugins-anthropic
./livekit-plugins/livekit-plugins-anthropic \
./livekit-plugins/livekit-plugins-llama-index

- name: Install stub packages
run: |
Expand Down
11 changes: 7 additions & 4 deletions .github/workflows/publish-docs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ name: Publish docs
on:
workflow_dispatch:
workflow_call:
secrets:
secrets:
DOCS_DEPLOY_AWS_ACCESS_KEY: {}
DOCS_DEPLOY_AWS_API_SECRET: {}

Expand All @@ -19,7 +19,7 @@ jobs:
- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: '3.12' # Specify the Python version you want to use
python-version: "3.12" # Specify the Python version you want to use

- name: Create and activate virtual environment
run: |
Expand All @@ -42,7 +42,10 @@ jobs:
./livekit-plugins/livekit-plugins-elevenlabs \
./livekit-plugins/livekit-plugins-google \
./livekit-plugins/livekit-plugins-nltk \
./livekit-plugins/livekit-plugins-openai
./livekit-plugins/livekit-plugins-openai \
./livekit-plugins/livekit-plugins-rag \
./livekit-plugins/livekit-plugins-silero \
./livekit-plugins/livekit-plugins-llama-index

- name: Build Docs
run: |
Expand All @@ -56,4 +59,4 @@ jobs:
env:
AWS_ACCESS_KEY_ID: ${{ secrets.DOCS_DEPLOY_AWS_ACCESS_KEY }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.DOCS_DEPLOY_AWS_API_SECRET }}
AWS_DEFAULT_REGION: "us-east-1"
AWS_DEFAULT_REGION: "us-east-1"
40 changes: 31 additions & 9 deletions livekit-plugins/livekit-plugins-llama-index/README.md
Original file line number Diff line number Diff line change
@@ -1,14 +1,36 @@
# LiveKit Plugins Minimal
# LiveKit Plugins Llama Index

This is a minimal example of a LiveKit plugin for Agents.
Agent Framework plugin for using Llama Index. Currently supports [Query Engine](https://docs.llamaindex.ai/en/stable/module_guides/deploying/query_engine/) and [Chat Engine](https://docs.llamaindex.ai/en/stable/module_guides/deploying/chat_engines/).

### Developer note
## Install

When copying this directory over to create a new `livekit-plugins` package, make sure it's nested within the `livekit-plugins` folder and that the `"name"` field in `package.json` follows the proper naming convention for CI:
```bash
pip install livekit-plugins-llama-index
```

## Query Engine

Query Engine is primarily used for RAG. See [example voice agent](https://github.com/livekit/agents/blob/main/examples/voice-pipeline-agent/llamaindex-rag/query_engine.py)

## Chat Engine

```json
{
"name": "livekit-plugins-<name>",
"private": true
}
Chat Engine can be used as an LLM within the frame work.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ChatEngine isn't the recommended way tho. Since this one doesn't support function calls (It is using the LLM from lama_index)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This one: https://github.com/livekit/agents/blob/main/examples/voice-pipeline-agent/llamaindex-rag/query_engine.py
and this one https://github.com/livekit/agents/blob/main/examples/voice-pipeline-agent/llamaindex-rag/retrieval.py
are more appropriate, but it depends on the needs.

The ChatEngine is also doing some "context" management (like keeping the previous rag search of the prior question etc.). They all have pros and cons

Copy link
Member Author

@davidzhao davidzhao Oct 16, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see, you have an example for it, I'm just documenting it. should we not point folks down that path?

```python
# load the existing index
storage_context = StorageContext.from_defaults(persist_dir=<mydir>)
index = load_index_from_storage(storage_context)
chat_engine = index.as_chat_engine(chat_mode=ChatMode.CONTEXT)

async def entrypoint(ctx: JobContext):
...
assistant = VoicePipelineAgent(
davidzhao marked this conversation as resolved.
Show resolved Hide resolved
vad=silero.VAD.load(),
stt=deepgram.STT(),
llm=llama_index.LLM(chat_engine=chat_engine),
tts=openai.TTS(),
chat_ctx=initial_ctx,
)
```

full example [here](https://github.com/livekit/agents/blob/main/examples/voice-pipeline-agent/llamaindex-rag/chat_engine.py)
Loading