Skip to content

Latest commit

 

History

History
62 lines (40 loc) · 1.71 KB

aiplayground.md

File metadata and controls

62 lines (40 loc) · 1.71 KB

NVIDIA AI Foundation

NVIDIA AI Foundation lets developers to experience state of the art LLMs accelerated by NVIDIA. Developers get free credits for 10K requests to any of the available models.

Prepare the environment

  1. Navigate to https://catalog.ngc.nvidia.com/ai-foundation-models.

  2. Find the Mixtral x7B model icon and click Learn More.

Diagram

  1. Select the API navigation bar and click on the Generate key option..

Diagram

  1. Save the generated API key.

Deploy

  1. Clone the Generative AI examples Git repository.

⚠️ NOTE: This example requires Git Large File Support (LFS)

$ sudo apt -y install git-lfs
$ git clone git@github.com:NVIDIA/GenerativeAIExamples.git
Cloning into 'GenerativeAIExamples'...
$ cd GenerativeAIExamples/
$ git lfs pull
  1. Add your NGC API key to compose.env to use the NVIDIA endpoint.
$ cd GenerativeAIExamples

$ grep NVIDIA_API_KEY deploy/compose/compose.env
 export NVIDIA_API_KEY="nvapi-*"
  1. Set the nv-ai-foundation example in compose.env.
 export RAG_EXAMPLE="nvidia_ai_foundation"
  1. Deploy the developer RAG example via Docker compose.
$ source deploy/compose/compose.env ; docker compose -f deploy/compose/docker-compose-nv-ai-foundation.yaml build

$ docker compose -f deploy/compose/docker-compose-nv-ai-foundation.yaml up -d

$ docker ps --format "table {{.ID}}\t{{.Names}}\t{{.Status}}"
CONTAINER ID   NAMES             STATUS
70ef27ae4c91   llm-playground    Up 56 seconds
4aacfbe89464   chain-server      Up 56 seconds

Test

  1. Follow steps 1 - 5 in the "Test" section of example 02.