Fack answers frequently asked questions using Generative AI. Here is an example:
At Salesforce, we use fack for internal Q&A. We feed documents and summarized slack threads to fack.
In some channels, we see up to 40% of our support requests receive helpful answers from fack.
We are experiementing with other use cases like:
- Improving incident resolution by quickly finding suggested solutions
- Translating between query languages
Almost any RAG-based solution can be quickly implmented in fack.
The term FAQ, or Frequently Asked Questions, is often pronounced fack.
Fack is built on the principles outlined by OpenAI for question/answer applications in documents like:
- https://platform.openai.com/docs/tutorials/web-qa-embeddings
- https://github.com/openai/openai-cookbook/blob/main/examples/Question_answering_using_embeddings.ipynb
Reuse your documents across multiple systems. Need a Q/A interface? Done. Need to provide answers in a chat bot? Done.
Working with LLMs is deceptively complex. Vector embeddings, token counting, effective prompting all require thought to make effective use of LLMs. By having a shared service, you can have one service to manage retrieval-augmented search and ask queries.
When managing thousands of documents, managing documents is difficult. Are there duplicates? What are the groups of documents?
All documents are not equal. You have different groups within your company and organization. Different teams need to manage their documents. Visibilty may be an issue.
- Open Source. This should be reusable as far as possible. Nothing in the tool should be hardcoded to Salesforce or any team in Salesforce. The tool has clear deployment and development instructions.
- API first. The end user interactions will likely happen through other bots and applications. So, the service should be API first.
- Multi-tenant. Different teams should be able to share the same technology, without mixing their data.
The OpenAI doc outlines the reasons why search/retrieval is typically better than fine-tuning.
- "Fine-tuning is better suited to teaching specialized tasks or styles, and is less reliable for factual recall."
- Search/retrival enables more reliable source citation. We can provide specific documents with urls/names to the GPT and require references to the documents used for answering the question. In fine-tuning, the original source is most likely lost or unavailable.
- Rails 7
- Ruby 3+
- Postgres with pg_vector support
- Clone the repo
git clone https://github.com/salesforce/fack.git
- Install dependencies
bundle
- Database creation/migration
rails db:create
rails db:migrate
TEST_PASSWORD='<your_password>' rails db:seed
Set TEST_PASSWORD to an 8+ character string with a number, uppercase letter and special character. '2Testai!' as an example.
- Create a .env file in the root directory. Provide these variables.
# LLM AUTH. You need to provide an OpenAI Key or a Salesforce Einstein Org
## OpenAI key from https://platform.openai.com/account/api-keys
OPENAI_API_KEY=<openai token>
## OR Salesforce Einstein credentials from your Org
SALESFORCE_CONNECT_ORG_URL=
SALESFORCE_CONNECT_CLIENT_ID=
SALESFORCE_CONNECT_CLIENT_SECRET=
SALESFORCE_CONNECT_USERNAME=
SALESFORCE_CONNECT_PASSWORD=
# i.e. https://fack.yourdomain.com or http://localhost:3000 Used to generate URLs in the answers.
ROOT_URL=
## SAML/SSO Metadata URL (OPTIONAL)
SSO_METADATA_URL=
## Max number of document tokens to send to the GPT prompt. (OPTIONAL)
MAX_PROMPT_DOC_TOKENS=
## Max tokens to send in the prompt (OPTIONAL, DEFAULT 10,000)
EGPT_MAX_TOKENS=
## Which OpenAI model to use. (OPTIONAL)
EGPT_GEN_MODEL=
## Disable Password Login if you have SSO enabled. (OPTIONAL)
DISABLE_PASSWORD_LOGIN=<true/false>
- Open a new terminal and start the Background job for AI Calls
rake jobs:work
You should see: [Worker(host:host.something.com pid:89737)] Starting job worker
- Start web server
rails s
Open http://localhost:3000 and login with admin@fack.com as the username and the TEST_PASSWORD you set earlier.
bundle exec rspec
The tests are located under the spec
directory.
Coming soon
- Definition
- Assistant format.
- User Input Prompt
- LLM prompt. up to 50k tokens.
- Libraries allowed
- Assistant format.
- Objects
- Assistant
- Session
- Message (content, from: <user, llm>)
- Operation in UI
- Click assistant of choice
- User is prompted for input from definition. Click start.
- Taken to Session page /assistant/1/session/3 with message shown.
- Fack sends user answer
- Adds relevant docs from libraries to the prompt
- Adds the custom prompts
- LLM can respond with the answer or a request user for more information
- User can continue to respond
- Fack sends user answer and updated docs from libraries to the AI
- Fack sends the prior user responses and ai questions
- Continues until user is satisfied.
- If the answer is good, the user can give that feedback to benefit future users.
POST
/api/v1/questions
Create a new Question (Ask AI for answer)
name type data type description question required text The question to ask of the documentation library_ids_included optional comma separated ids (reference to library) The libraries to limit the answers
http code content-type response 201
text/plain;charset=UTF-8
JSON Object 400
application/json
{"code":"400","message":"Bad Request"}
curl -X POST -H "Authorization: Bearer <token>" -H "Content-Type: application/json" -d '{"question": { "question" : "how do i setup falcon?", library_ids_included: ["1"] }}' http://localhost:3000/api/v1/questions
{ "id": 226, "question": "how do i setup falcon?", "status":"generating", "answer": "# ANSWER\nTo set up Falcon, you need to install the Falcon CLI. Here are the steps to install the Falcon CLI:\n\n1. For macOS users, install the Falcon CLI with brew:\n ```\n brew tap sfdc-falcon/cli git[@git.soma.salesforce.com:sfdc-falcon/>homebrew-cli.git](https://git.soma.salesforce.com/git.soma.salesforce.com:sfdc-falcon/homebrew-cli.git)\n brew install falcon-cli\n ```\n\n2. For Linux users, install the Falcon CLI with `curl`:\n ```\n curl -sSL https://sfdc.co/get-falcon-cli | bash\n ```\n\n3. Verify that you've successfully installed the CLI by logging in:\n ```\n falcon login\n ```\n\nYou can find more information about setting up the Falcon CLI in the [Install the Falcon Command Line Interface (CLI)](https://git.soma.salesforce.com/tech-enablement/falcon-paved-path/blob/main/install-falcon-cli.md) document.\n\n# SOURCES\n- [Install the Falcon Command Line Interface (CLI)](https://git.soma.salesforce.com/tech-enablement/falcon-paved-path/blob/main/install-falcon-cli.md)", "created_at": "2023-11-03T17:28:43.625Z", "updated_at": "2023-11-03T17:28:43.625Z", "library_ids_included": ["1"], "url": "http://localhost:3000/questions/226.json" }
GET
/api/v1/questions/_id_
Retrieve Question
name type data type description
name type data type description question text The question to ask of the documentation status pending, generating, generated, failed The status of the generated answer. Poll every 5 seconds until the status is generated or failed. able_to_answer boolean Was the GPT able to generate an answer? (true/false) url text URL to access the question answer text Answer to the question
http code content-type response 200
text/plain;charset=UTF-8
400
application/json
{"code":"400","message":"Bad Request"}
curl -X GET -H "Authorization: Bearer <token>" http://localhost:3000/api/v1/questions/226
{ "id": 226, "question": "how do i setup falcon?", "status":"generated", "answer": "# ANSWER\nTo set up Falcon, you need to install the Falcon CLI. Here are the steps to install the Falcon CLI:\n\n1. For macOS users, install the Falcon CLI with brew:\n ```\n brew tap sfdc-falcon/cli git[@git.soma.salesforce.com:sfdc-falcon/>homebrew-cli.git](https://git.soma.salesforce.com/git.soma.salesforce.com:sfdc-falcon/homebrew-cli.git)\n brew install falcon-cli\n ```\n\n2. For Linux users, install the Falcon CLI with `curl`:\n ```\n curl -sSL https://sfdc.co/get-falcon-cli | bash\n ```\n\n3. Verify that you've successfully installed the CLI by logging in:\n ```\n falcon login\n ```\n\nYou can find more information about setting up the Falcon CLI in the [Install the Falcon Command Line Interface (CLI)](https://git.soma.salesforce.com/tech-enablement/falcon-paved-path/blob/main/install-falcon-cli.md) document.\n\n# SOURCES\n- [Install the Falcon Command Line Interface (CLI)](https://git.soma.salesforce.com/tech-enablement/falcon-paved-path/blob/main/install-falcon-cli.md)", "created_at": "2023-11-03T17:28:43.625Z", "updated_at": "2023-11-03T17:28:43.625Z", "url": "http://localhost:3000/questions/226.json" }
GET
/api/v1/questions
List Questions
name | type | data type | description |
---|---|---|---|
page | optional | integer | The page number to retrieve. Defaults to 1. |
name | type | data type | description |
---|---|---|---|
questions | array | An array of question objects, each containing question details |
Each object in the questions
array includes:
name | type | data type | description |
---|---|---|---|
id | integer | The ID of the question | |
question | text | The content of the question | |
status | text | The status of the question (e.g., generated) | |
answer | text | The answer to the question | |
url | text | URL to access the question | |
created_at | datetime | The creation date and time of the question | |
updated_at | datetime | The last update date and time of the question |
http code | content-type | response |
---|---|---|
200 |
application/json |
JSON array of questions |
400 |
application/json |
{"code":"400","message":"Bad Request"} |
curl -X GET -H "Authorization: Bearer <token>" "http://localhost:3000/api/v1/questions"
{ "questions": [ { "id": 226, "question": "how do i setup a new service?", "status": "generated", "answer": "# ANSWER\nTo set up ...", "created_at": "2023-11-03T17:28:43.625Z", "updated_at": "2023-11-03T17:28:43.625Z", "url": "http://localhost:3000/questions/226.json" }, { "id": 227, "question": "how do i use gen ai?", "status": "generated", "answer": "# ANSWER\n...", "url": "http://localhost:3000/questions/227.json", "created_at": "2023-11-14T01:55:11.731Z", "updated_at": "2024-03-01T22:58:55.865Z" } ] }
POST
/api/v1/documents
Create a new Document
name type data type description document required text The content of the document. 10,000 token limit. title required text The title of the document library_id required text The ID of the library to which this document will be added external_id optional text A unique ID provided by the client. If a POST request includes the same external_id as an existing record, the record will be updated instead of created.
Make sure you have the top level "document" attribute.
"document": { "document": "Restart your computer to fix it.", "title": "How to fix a computer", "library_id": 23, }
http code content-type response 201
text/plain;charset=UTF-8
Document created successfully
400
application/json
{"code":"400","message":"Bad Request"}
curl -X POST -H "Content-Type: application/json" -H "Authorization: Bearer <token>" -d '{"document": {"document":"Document Content", "library_id":"your_library_id", "external_id":"optional_unique_id"}}' http://localhost:3000/api/v1/documents
GET
/api/v1/documents/_id_
Retrieve Document
name type data type description
name | type | data type | description |
---|---|---|---|
document | object | A document objects |
The document
object includes:
name | type | data type | description |
---|---|---|---|
id | integer | The ID of the document | |
document | text | The content of the document | |
title | text | The title of the document | |
url | text | URL to access the document | |
created_at | datetime | The creation date and time of the document | |
updated_at | datetime | The last update date and time of the document |
http code content-type response 200
text/plain;charset=UTF-8
400
application/json
{"code":"400","message":"Bad Request"}
curl -X GET -H "Authorization: Bearer <token>" http://localhost:3000/api/v1/documents/<id>
{ "id": 1, "document": "# QUESTION\nHow do I use gen ai?\n\n# ANSWER\n...", "title": "How do I use GenAI?", "url": "http://localhost:3000/documents/1", "length": 97, "created_at": "2023-11-14T01:55:11.731Z", "updated_at": "2024-03-01T22:58:55.865Z" }
GET
/api/v1/documents
List Documents
name | type | data type | description |
---|---|---|---|
page | optional | integer | The page number to retrieve. Defaults to 1. |
name | type | data type | description |
---|---|---|---|
documents | array | An array of document objects, each containing document details |
Each object in the documents
array includes:
name | type | data type | description |
---|---|---|---|
id | integer | The ID of the document | |
document | text | The content of the document | |
title | text | The title of the document | |
url | text | URL to access the document | |
created_at | datetime | The creation date and time of the document | |
updated_at | datetime | The last update date and time of the document |
http code | content-type | response |
---|---|---|
200 |
application/json |
JSON array of documents |
400 |
application/json |
{"code":"400","message":"Bad Request"} |
curl -X GET -H "Authorization: Bearer <token>" "http://localhost:3000/api/v1/documents?page=1"
{ "documents": [ { "id": 1, "document": "# QUESTION\nHow do I use gen ai?\n\n# ANSWER\nThere is no direct answer provided in the conversation.", "url": "http://localhost:3000/documents/1", "created_at": "2023-11-14T01:55:11.731Z", "updated_at": "2024-03-01T22:58:55.865Z" }, { "id": 2, "document": "# QUESTION\nHow do I integrate API?\n\n# ANSWER\nTo integrate an API, first identify the API you need to integrate with...", "url": "http://localhost:3000/documents/2", "created_at": "2023-12-14T02:55:11.731Z", "updated_at": "2024-01-02T23:58:55.865Z" } ... ] }
POST
/api/v1/libraries
Create a new Library
name type data type description name required text The name of the library
http code content-type response 201
text/plain;charset=UTF-8
Library created successfully
400
application/json
{"code":"400","message":"Bad Request"}
curl -X POST -H "Content-Type: application/json" -H "Authorization: Bearer <token>" -d '{"name":"Library Name"}' http://localhost:3000/api/v1/libraries
GET
/api/v1/library/_id_
Retrieve Library
name type data type description
http code content-type response 200
text/plain;charset=UTF-8
400
application/json
{"code":"400","message":"Bad Request"}
curl -X GET -H "Authorization: Bearer <token>" http://localhost:3000/api/v1/libraries/<id>
{ "id": 1, "name": "My Docs", "created_at": "2023-11-15T20:17:25.665Z", "updated_at": "2023-12-01T19:59:44.618Z", "url": "http://localhost:3000/libraries/1" }
GET
/api/v1/libraries
List Libraries
name | type | data type | description |
---|---|---|---|
page | optional | integer | The page number to retrieve. Defaults to 1. |
name | type | data type | description |
---|---|---|---|
libraries | array | An array of library objects, each containing library details |
Each object in the documents
array includes:
name | type | data type | description |
---|---|---|---|
id | integer | The ID of the library | |
name | text | Name of the library | |
created_at | datetime | The creation date and time of the library | |
updated_at | datetime | The last update date and time of the library |
http code | content-type | response |
---|---|---|
200 |
application/json |
JSON array of libraries |
400 |
application/json |
{"code":"400","message":"Bad Request"} |
curl -X GET -H "Authorization: Bearer <token>" "http://localhost:3000/api/v1/libraries"
{ "libraries": [ { "id": 1, "name": "My Library", "created_at": "2023-11-14T01:55:11.731Z", "updated_at": "2024-03-01T22:58:55.865Z" }, { "id": 2, "name": "My Second Library", "created_at": "2023-01-14T01:55:11.731Z", "updated_at": "2023-03-01T22:58:55.865Z" }, ... ] }
- Get the SSO Metadata URL from your SSO provider. i.e. https://company.okta.com/app/xyz/sso/saml/metadata
- Set the SSO_METADATA_URL to the url from previous step in your .env file or environment.
- Restart your app.
- Optionally, disable username/password login with DISABLE_PASSWORD_LOGIN=true environment variable.
The library is a collection of similar documents. For example:
- Infrastructure
- Dev Docs
- Onboarding
The library allows document owners to keep their documents separate from each other. It also enables more selective question/answers.
- Go to /libraries in the UI
- Locate the library you want.
- Note the id in the URL.
- Go to /libraries in the UI
- Click "New Library" button.
- Provide a name and save. Get the id of the library from the URL.
- Open /api_tokens in the ui.
- Create a token.
You can clone your doc repo or have an directory anywhere on your computer.
IMPORT_API_TOKEN=<your token>
ROOT_URL=<root url> # "http://localhost:3000" if you are testing locally.
jq is a dependency.
See the download page for instructions.
./import.sh -l <library_id> -d <path_to_your_docs>
- Go to /libraries and click on the library you used.
- See if the documents from your import are listed.