new Error("Please install chromadb as a dependency with, e.g. `npm install -S chromadb`") but chromadb existed in node_modules in newer langchain since version 147 #2637

helxsz · 2023-09-14T23:47:28Z

My previous code used to be working in 0.0.145,

import { CheerioWebBaseLoader } from "langchain/document_loaders/web/cheerio";
import { RecursiveCharacterTextSplitter } from "langchain/text_splitter";
import { HuggingFaceTransformersEmbeddings } from "langchain/embeddings/hf_transformers

const huggerEmbedding = new HuggingFaceTransformersEmbeddings({
  modelName: "Xenova/all-MiniLM-L6-v2",
});

import { Chroma } from "langchain/vectorstores/chroma";

const loader = new CheerioWebBaseLoader(
  "https://lilianweng.github.io/posts/2023-06-23-agent/"
);
const docs = await loader.load();

const splitter = new RecursiveCharacterTextSplitter({
  chunkOverlap: 0,
  chunkSize: 500,
});

const splitDocuments = await splitter.splitDocuments(docs);

const vectorStore = await Chroma.fromDocuments(splitDocuments, huggerEmbedding, {
  collectionName: "a-test-collection",
  url: "http://localhost:9007", // Optional, will default to this value
  collectionMetadata: {
    "hnsw:space": "cosine",
  }
});

const retrievedDocs = await vectorStore.similaritySearch(
  "What are the approaches to Task Decomposition?"
);

now in newer version, 0.0.147, 0.0.148, 0.0.149(not tested yet but seems to be still wrong) are all getting an errors saying

node_modules/_langchain@0.0.147@langchain/dist/vectorstores/chroma.js:287
            throw new Error("Please install chromadb as a dependency with, e.g. `npm install -S chromadb`");
                  ^
Error: Please install chromadb as a dependency with, e.g. `npm install -S chromadb`
    at Function.imports (file:///Users/xisizhe/Documents/my_projects/langchain/lang147/node_modules/_langchain@0.0.147@langchain/dist/vectorstores/chroma.js:287:19)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async Chroma.ensureCollection (file:///Users/xisizhe/Documents/my_projects/langchain/lang147/node_modules/_langchain@0.0.147@langchain/dist/vectorstores/chroma.js:89:42)
    at async Chroma.addVectors (file:///Users/xisizhe/Documents/my_projects/langchain/lang147/node_modules/_langchain@0.0.147@langchain/dist/vectorstores/chroma.js:126:28)
    at async Function.fromDocuments (file:///Users/xisizhe/Documents/my_projects/langchain/lang147/node_modules/_langchain@0.0.147@langchain/dist/vectorstores/chroma.js:262:9)
    at async file:///Users/xisizhe/Documents/my_projects/langchain/lang147/embedding_huggerface2.ts:31:21

however the node_modules has the latest chromadb 1.5.8 , even downgraded to 1.5.3, still output the same error for no reason, any ideas why chromadb package not found ?

The text was updated successfully, but these errors were encountered:

dosubot · 2023-09-14T23:56:04Z

🤖

Hello,

I understand that you're having an issue with the chromadb dependency in the newer versions of langchainjs. Based on the error message you're receiving, it seems like the chromadb module is not being found, even though it exists in your node_modules.

From the context provided, there are two similar issues that have been resolved in the past:

Update chroma integration to support 1.5.0: This issue was resolved by merging and releasing a PR. You can check the changes made in this PR to see if they apply to your situation.
Allow ChromaClient to be initialized outside Chroma Vector Store: This issue was resolved by making changes in a commit. You can follow the changes made in the commit Fix chroma require left around from before ESM conversion #220 to resolve your issue.

If these solutions do not resolve your issue, it might be helpful to check the following:

Ensure that chromadb is correctly installed in your project. You can verify this by running npm list chromadb in your project directory. This should return the version of chromadb installed in your project.
If chromadb is installed correctly, the issue might be with the way langchainjs is trying to import it. In the newer versions of langchainjs, there might be changes in the way dependencies are handled. You can check the langchainjs documentation or the source code to see if there are any changes in this regard.
If the above steps do not help, you might want to consider creating a minimal reproduction of the issue and sharing it. This would help in identifying the exact cause of the issue.

I hope this helps. Let me know if you have any other questions or if there's anything else I can do to assist you.

Sources

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

NajamShehzad · 2023-09-17T18:47:21Z

Getting same error while using chromadb

HenryHengZJ · 2023-09-18T11:41:15Z

bump, getting the same errors as well @jacoblee93 any idea?

jacoblee93 · 2023-09-18T13:51:39Z

Will have a look today, thanks for the heads up!

pooriaarab · 2023-09-18T18:28:38Z

@jacoblee93 thx. Is there an ETA on when it can get fixed?

Currently Flowise is in production with Chroma and can't use it 🫠

jacoblee93 · 2023-09-18T19:00:40Z

@jeffchuber am about to dig in but any idea what could have changed?

We set our Chroma peer dep to * so you should in theory be able to downgrade.

jacoblee93 · 2023-09-18T19:09:19Z

@pooriaarab @NajamShehzad @HenryHengZJ @helxsz hard locking Chroma to 1.5.3 (no caret) fixed it for me:

{
  "name": "testproject",
  "version": "1.0.0",
  "description": "",
  "type": "module",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "author": "",
  "license": "ISC",
  "dependencies": {
    "@xenova/transformers": "^2.6.1",
    "cheerio": "^1.0.0-rc.12",
    "chromadb": "1.5.3",
    "langchain": "^0.0.150"
  }
}

If you add a caret it'll automatically upgrade to 1.5.9, which is likely where something bad was introduced.

@jeffchuber may have more insight.

pooriaarab · 2023-09-18T19:11:50Z

@jacoblee93 thx for the quick update.

this is from the Chroma discord:
https://discord.com/channels/1073293645303795742/1152647545122738286/1153294473690873906

Apparently, if you rhard lock and rollback to Chroma 1.5.6 it will work

pooriaarab · 2023-09-18T19:30:39Z

BUMP

@jacoblee93 @jeffchuber i hard locked Chroma but get this error now:

Chroma getOrCreateCollection error: Error: TypeError: fetch failed

jacoblee93 · 2023-09-18T19:52:13Z

Maybe if you go to 1.5.8?

tazarov · 2023-09-18T20:11:59Z

Ideally, this would be fixed with chroma-core/chroma#1153

Tested with LC@0.0.150

also works with LC@0.0.147 (which I think is what latest flowise is using)

jeffchuber · 2023-09-19T04:22:58Z

I just released 1.5.10 - thanks to @tazarov for the assistance. this temporarily removes the deps on xeonova/transformers and webAI until we can figure out how to package those libraries effectively

jeffchuber · 2023-09-19T04:31:36Z

I confirmed with a minimal example that everything should be playing nice together - can folks give this a try?

pooriaarab · 2023-09-19T15:46:27Z

@jeffchuber issue still exists. @tazarov mentioned 1.5.11 version of Chroma can fix the dependency errors.

jacoblee93 · 2023-09-19T19:54:41Z

Sounds like it fixed it! Thanks all for reporting - hopefully this doesn't happen again.

martin-242 · 2023-09-19T20:53:41Z

I have latest flowise and chroma 1.5.11 and it still showing the same error...

jacoblee93 · 2023-09-19T23:31:00Z

@jeffchuber any idea what's going on?

jacoblee93 · 2023-09-19T23:34:24Z

FWIW langchain + chroma seems to work for me with npm

helxsz · 2023-09-25T15:24:25Z

@jacoblee93 hi, lee, I have tried your solution on

  "chromadb": "1.5.3",
   "langchain": "^0.0.150"'

the issue of chromadb dependency is gone, but it seems I can't use the local transformer model as I did

import { pipeline, env } from '@xenova/transformers';
env.localModelPath = '/Users/xisizhe/Desktop';
// @ts-ignore
env.allowRemoteModels = false;
env.backends.onnx.wasm.wasmPaths = '/Users/test//Xenova/ort/';

the error is

TypeError: fetch failed
    at Object.fetch (node:internal/deps/undici/undici:11457:11)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
    at runNextTicks (node:internal/process/task_queues:64:3)
    at processImmediate (node:internal/timers:447:9)
    at async getModelFile (file:///Users/test/Documents/my_projects/langchain/lang149/node_modules/_@xenova_transformers@2.6.1@@xenova/transformers/src/utils/hub.js:468:24)
    at async getModelJSON (file:///Users/test/Documents/my_projects/langchain/lang149/node_modules/_@xenova_transformers@2.6.1@@xenova/transformers/src/utils/hub.js:542:18)

looks like it always try to fetch the hub for the model.

jacoblee93 · 2023-09-25T15:37:44Z

Doesn't look like you're using it through LangChain?

We only support @xenova/transformers embeddings models right now:

https://js.langchain.com/docs/modules/data_connection/text_embedding/integrations/transformers

helxsz · 2023-09-26T06:20:22Z

This is exact example I am testing right now.

import { pipeline, env } from '@xenova/transformers';
import * as fs from "fs"
env.localModelPath = '/Users/test/Desktop';
// @ts-ignore
env.allowRemoteModels = false;
env.backends.onnx.wasm.wasmPaths = '/Users/test/Desktop/Xenova/ort/';

import { HuggingFaceTransformersEmbeddings } from "langchain/embeddings/hf_transformers";

const model = new HuggingFaceTransformersEmbeddings({
  modelName: "Xenova/all-MiniLM-L6-v2",
});

/* Embed queries */
const res = await model.embedQuery(
  "What would be a good company name for a company that makes colorful socks?"
);
console.log({ res });
/* Embed documents */
const documentRes = await model.embedDocuments(["Hello world", "Bye bye"]);
console.log({ documentRes });

langchain 0.0147 can use the local model whereas langchain 0.0.150 looking to fetch the model from huggerface hub

helxsz · 2023-09-26T06:55:30Z

This is exact example I am testing right now.

import { pipeline, env } from '@xenova/transformers';
import * as fs from "fs"
env.localModelPath = '/Users/test/Desktop';
// @ts-ignore
env.allowRemoteModels = false;
env.backends.onnx.wasm.wasmPaths = '/Users/test/Desktop/Xenova/ort/';

import { HuggingFaceTransformersEmbeddings } from "langchain/embeddings/hf_transformers";

const model = new HuggingFaceTransformersEmbeddings({
  modelName: "Xenova/all-MiniLM-L6-v2",
});

/* Embed queries */
const res = await model.embedQuery(
  "What would be a good company name for a company that makes colorful socks?"
);
console.log({ res });
/* Embed documents */
const documentRes = await model.embedDocuments(["Hello world", "Bye bye"]);
console.log({ documentRes });

langchain 0.0147 can use the local model whereas langchain 0.0.150 looking to fetch the model from huggerface hub

ricardomatias · 2024-03-13T16:00:10Z

This is still happening with langchain (^0.1.27) and chromadb (^1.8.1).

Error: Please install chromadb as a dependency with, e.g. `npm install -S chromadb`
    at t.imports (file:///X/Projects/js/ai/live-llm/dist/index.js:238:10239)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async t.ensureCollection (file:///X/Projects/js/ai/live-llm/dist/index.js:238:7908)
    at async t.addVectors (file:///X/Projects/js/ai/live-llm/dist/index.js:238:8600)
    at async t.fromDocuments (file:///X/Projects/js/ai/live-llm/dist/index.js:238:9990)
    at async file:///X/Projects/js/ai/live-llm/dist/index.js:245:473

jacoblee93 · 2024-03-13T16:40:49Z

@bracesproul you were trying this recently weren't you?

jacoblee93 · 2024-03-13T16:44:42Z

Can you give me a bit of information about your environment @ricardomatias? It looks like it might be something like Next.js?

george-sagaiq · 2024-12-19T04:12:49Z

Hi All - I know this issue is closed, but I recently came across this problem in December 2024 and found a fix/work-around for it. I was experiencing this issue when using LangChain's Chroma libraries for unit tests with testcontainers. Changing my npm test command from:

"test": "jest"

to:

"test": "node --experimental-vm-modules ../../node_modules/jest/bin/jest.js"

fixed the problem. Please note the ../../ is specific to the overall structure of my package layout in my nx-based project.

Here is a working jest test suite that uses testcontainers to spin up a chromadb instance, and then the test uses both the LangChain chroma abstractions in addition to the ChromaClient directly provided by the chroma team.

import { ChromaDBContainer, StartedChromaDBContainer } from '@testcontainers/chromadb';
import { ChromaClient, OllamaEmbeddingFunction } from 'chromadb';
import { OllamaEmbeddings } from '@langchain/ollama';
import { PDFLoader } from '@langchain/community/document_loaders/fs/pdf';
import { Chroma } from '@langchain/community/vectorstores/chroma';
import { RecursiveCharacterTextSplitter } from 'langchain/text_splitter';

// eslint-disable-next-line @typescript-eslint/no-var-requires
const path = require('path');

jest.setTimeout(10000);

const CHROMA_IMAGE = 'chromadb/chroma:0.5.18';

describe('chroma', () => {
  let chromaDbContainer: StartedChromaDBContainer | undefined = undefined;
  let chroma: ChromaClient | undefined = undefined;
  let chromaEmbeddingFunction: OllamaEmbeddingFunction | undefined = undefined;

  beforeAll(async () => {
    chromaDbContainer = await new ChromaDBContainer(CHROMA_IMAGE).start();

    chroma = new ChromaClient({
      path: `http://${chromaDbContainer.getHost()}:${chromaDbContainer.getMappedPort(8000)}`
    });

    chromaEmbeddingFunction = new OllamaEmbeddingFunction({
      url: `http://localhost:11434/api/embeddings`,
      model: 'nomic-embed-text'
    });
  });

  afterAll(async () => {
    if (chromaDbContainer) {
      await chromaDbContainer.stop();
    }
  });

  it('Just Works™', async () => {
    const documentPath = path.normalize('test-data/sample.pdf');
    const loader = new PDFLoader(documentPath, {
      splitPages: true
    });
    const textSplitter = new RecursiveCharacterTextSplitter({
      chunkSize: 300,
      chunkOverlap: 50
    });
    const doc = await loader.load();
    const splitDoc = await textSplitter.splitDocuments(doc);

    const ollamaEmbeddings = new OllamaEmbeddings({
      baseUrl: 'http://localhost:11434',
      model: 'nomic-embed-text'
    });

    const collectionName = 'test-collection';

    // ingest using the LangChain chroma abstraction
    await Chroma.fromDocuments(splitDoc, ollamaEmbeddings, {
      collectionName,
      url: `http://${chromaDbContainer.getHost()}:${chromaDbContainer.getMappedPort(8000)}`
    });

    // retrieve using the native chroma client
    const assistantCollection = await chroma.getCollection({ name: collectionName, embeddingFunction: chromaEmbeddingFunction });
    const collectionContents = await assistantCollection.get();
    expect(collectionContents.documents.length).toBe(93);
  });
});

dosubot bot added the auto:bug Related to a bug, vulnerability, unexpected error with an existing feature label Sep 14, 2023

jacoblee93 self-assigned this Sep 18, 2023

jacoblee93 closed this as completed Sep 19, 2023

jacoblee93 reopened this Sep 19, 2023

HenryHengZJ mentioned this issue Sep 20, 2023

Bugfix/Chroma installation fix FlowiseAI/Flowise#950

Merged

jacoblee93 closed this as completed Sep 25, 2023

This was referenced Dec 19, 2024

Issue with chroma on vercel production #3485

Closed

error: Error: Please install chromadb as a dependency with, e.g. npm install -S chromadb davideuler/gpt4-pdf-chatbot-langchain-chromadb#3

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

new Error("Please install chromadb as a dependency with, e.g. `npm install -S chromadb`") but chromadb existed in node_modules in newer langchain since version 147 #2637

new Error("Please install chromadb as a dependency with, e.g. `npm install -S chromadb`") but chromadb existed in node_modules in newer langchain since version 147 #2637

helxsz commented Sep 14, 2023

dosubot bot commented Sep 14, 2023

NajamShehzad commented Sep 17, 2023

HenryHengZJ commented Sep 18, 2023

jacoblee93 commented Sep 18, 2023

pooriaarab commented Sep 18, 2023

jacoblee93 commented Sep 18, 2023 •

edited

Loading

jacoblee93 commented Sep 18, 2023 •

edited

Loading

pooriaarab commented Sep 18, 2023

pooriaarab commented Sep 18, 2023

jacoblee93 commented Sep 18, 2023

tazarov commented Sep 18, 2023 •

edited

Loading

jeffchuber commented Sep 19, 2023

jeffchuber commented Sep 19, 2023

pooriaarab commented Sep 19, 2023

jacoblee93 commented Sep 19, 2023

martin-242 commented Sep 19, 2023

jacoblee93 commented Sep 19, 2023

jacoblee93 commented Sep 19, 2023

helxsz commented Sep 25, 2023 •

edited

Loading

jacoblee93 commented Sep 25, 2023

helxsz commented Sep 26, 2023

helxsz commented Sep 26, 2023

ricardomatias commented Mar 13, 2024

jacoblee93 commented Mar 13, 2024

jacoblee93 commented Mar 13, 2024

george-sagaiq commented Dec 19, 2024

new Error("Please install chromadb as a dependency with, e.g. npm install -S chromadb") but chromadb existed in node_modules in newer langchain since version 147 #2637

new Error("Please install chromadb as a dependency with, e.g. npm install -S chromadb") but chromadb existed in node_modules in newer langchain since version 147 #2637

Comments

helxsz commented Sep 14, 2023

dosubot bot commented Sep 14, 2023

Sources

NajamShehzad commented Sep 17, 2023

HenryHengZJ commented Sep 18, 2023

jacoblee93 commented Sep 18, 2023

pooriaarab commented Sep 18, 2023

jacoblee93 commented Sep 18, 2023 • edited Loading

jacoblee93 commented Sep 18, 2023 • edited Loading

pooriaarab commented Sep 18, 2023

pooriaarab commented Sep 18, 2023

jacoblee93 commented Sep 18, 2023

tazarov commented Sep 18, 2023 • edited Loading

jeffchuber commented Sep 19, 2023

jeffchuber commented Sep 19, 2023

pooriaarab commented Sep 19, 2023

jacoblee93 commented Sep 19, 2023

martin-242 commented Sep 19, 2023

jacoblee93 commented Sep 19, 2023

jacoblee93 commented Sep 19, 2023

helxsz commented Sep 25, 2023 • edited Loading

jacoblee93 commented Sep 25, 2023

helxsz commented Sep 26, 2023

helxsz commented Sep 26, 2023

ricardomatias commented Mar 13, 2024

jacoblee93 commented Mar 13, 2024

jacoblee93 commented Mar 13, 2024

george-sagaiq commented Dec 19, 2024

new Error("Please install chromadb as a dependency with, e.g. `npm install -S chromadb`") but chromadb existed in node_modules in newer langchain since version 147 #2637

new Error("Please install chromadb as a dependency with, e.g. `npm install -S chromadb`") but chromadb existed in node_modules in newer langchain since version 147 #2637

jacoblee93 commented Sep 18, 2023 •

edited

Loading

jacoblee93 commented Sep 18, 2023 •

edited

Loading

tazarov commented Sep 18, 2023 •

edited

Loading

helxsz commented Sep 25, 2023 •

edited

Loading