Skip to content

Latest commit

 

History

History
332 lines (272 loc) · 5.4 KB

english.md

File metadata and controls

332 lines (272 loc) · 5.4 KB

RKLLama REST API Documentation

Base URL

http://localhost:8080/

Quick Reference

Key Commands:

  • List available models: GET /models
  • Load a model: POST /load_model
  • Unload the current model: POST /unload_model
  • Get the current loaded model: GET /current_model
  • Generate output: POST /generate
  • Download a model from Hugging Face: POST /pull
  • Delete a model: POST /rm

API Endpoints

1. GET /models

Description

Returns the list of available models in the ~/RKLLAMA/models directory.

Request

GET /models

Response

  • 200 OK: List of available models.

    {
  "models": [
    "model1.rkllm",
    "model2.rkllm",
    "model3.rkllm"
  ]
}

  • 500 Internal Server Error: Directory not found.

    {
  "error": "The directory ~/RKLLAMA/models is not found."
}

Example

curl -X GET http://localhost:8080/models

2. POST /load_model

Description

Loads a specific model into memory.

Request

POST /load_model
Content-Type: application/json
Parameters
{
  "model_name": "model_name.rkllm"
}

Response

  • 200 OK: Model loaded successfully.

    {
  "message": "Model <model_name> loaded successfully."
}

  • 400 Bad Request: A model is already loaded or parameters are missing.

    {
  "error": "A model is already loaded. Please unload it first."
}

  • 400 Bad Request: Model not found.

    {
  "error": "Model <model_name> not found in the /models directory."
}

Example

curl -X POST http://localhost:8080/load_model \
-H "Content-Type: application/json" \
-d '{"model_name": "model1.rkllm"}'

3. POST /unload_model

Description

Unloads the currently loaded model.

Request

POST /unload_model

Response

  • 200 OK: Success.

    {
  "message": "Model unloaded successfully."
}

  • 400 Bad Request: No model is loaded.

    {
  "error": "No model is currently loaded."
}

Example

curl -X POST http://localhost:8080/unload_model

4. GET /current_model

Description

Returns the name of the currently loaded model.

Request

GET /current_model

Response

  • 200 OK: Success.

    {
  "model_name": "model_name"
}

  • 404 Not Found: No model is loaded.

    {
  "error": "No model is currently loaded."
}

Example

curl -X GET http://localhost:8080/current_model

5. POST /generate

Description

Generates a response using the loaded model.

Request

POST /generate
Content-Type: application/json
Parameters
{
  "messages": "prompt or chat_template",
  "stream": true
}

Response

  • 200 OK: Generated response.

    {
  "id": "rkllm_chat",
  "object": "rkllm_chat",
  "created": null,
  "choices": [{
    "role": "assistant",
    "content": "rkllama_output",
    "finish_reason": "stop"
  }],
  "usage": {
    "prompt_tokens": null,
    "completion_tokens": null,
    "total_tokens": null
  }
}

  • 400 Bad Request: No model is loaded.

    {
  "error": "No model is currently loaded."
}

Example

curl -X POST http://localhost:8080/generate \
-H "Content-Type: application/json" \
-d '{"messages": "Hello, how are you?", "stream": false}'

6. POST /pull

Description

Downloads and installs a model from Hugging Face.

Request

POST /pull
Content-Type: application/json
Parameters
{
  "model": "hf/username/repo/file.rkllm"
}

Response

  • 200 OK: Download in progress.
Downloading <file> (<size> MB)...
<progress>%
  • 400 Bad Request: Download error.
Error during download: <error>

Example

curl -X POST http://localhost:8080/pull \
-H "Content-Type: application/json" \
-d '{"model": "hf/username/repo/file.rkllm"}'

7. DELETE /rm

Description

Deletes a specific model.

Request

POST /rm
Content-Type: application/json
Parameters
{
  "model": "model_name.rkllm"
}

Response

  • 200 OK: Success.

    {
  "message": "The model has been successfully deleted."
}

  • 404 Not Found: Model not found.

    {
  "error": "The model: {model} cannot be found."
}

Example

curl -X DELETE http://localhost:8080/rm \
-H "Content-Type: application/json" \
-d '{"model": "model1.rkllm"}'

8. GET /

Description

Displays a welcome message and a link to the GitHub project.

Response

  • 200 OK: 
    {
  "message": "Welcome to RK-LLama!",
  "github": "https://github.com/notpunhnox/rkllama"
}

Example

curl -X GET http://localhost:8080/

Error Handling

  • 400: Bad Request due to incorrect parameters.
  • 404: Resource not found.
  • 500: Internal server error.

Practical Tips

  • Parameter Validation: Always double-check model names and file paths.
  • Troubleshooting: Check server logs for more details on internal errors.