Improve langchain4J integration

[emmanuelbernard]

2024-07-22 18:24:02,309 ERROR [io.qua.dep.dev.IsolatedDevModeMain] (main) Failed to start quarkus: java.lang.RuntimeException: io.quarkus.builder.BuildException: Build failure: Build failed due to errors
        [error]: Build step io.quarkus.arc.deployment.ArcProcessor#validate threw an exception: jakarta.enterprise.inject.spi.DeploymentException: jakarta.enterprise.inject.UnsatisfiedResolutionException: Unsatisfied dependency for type dev.langchain4j.store.embedding.inmemory.InMemoryEmbeddingStore and qualifiers [@Default]
        - injection target: com.emmanuelbernard.bearllm.Ingestor#store
        - declared on CLASS bean [types=[com.emmanuelbernard.bearllm.Ingestor, java.lang.Object], qualifiers=[@Default, @Any], target=com.emmanuelbernard.bearllm.Ingestor]
        at io.quarkus.arc.processor.BeanDeployment.processErrors(BeanDeployment.java:1546)
        at io.quarkus.arc.processor.BeanDeployment.init(BeanDeployment.java:338)
        at io.quarkus.arc.processor.BeanProcessor.initialize(BeanProcessor.java:167)
        at io.quarkus.arc.deployment.ArcProcessor.validate(ArcProcessor.java:490)
        at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
        at java.base/java.lang.reflect.Method.invoke(Method.java:580)
        at io.quarkus.deployment.ExtensionLoader$3.execute(ExtensionLoader.java:849)
        at io.quarkus.builder.BuildContext.run(BuildContext.java:256)
        at org.jboss.threads.ContextHandler$1.runWith(ContextHandler.java:18)
        at org.jboss.threads.EnhancedQueueExecutor$Task.doRunWith(EnhancedQueueExecutor.java:2516)
        at org.jboss.threads.EnhancedQueueExecutor$Task.run(EnhancedQueueExecutor.java:2495)
        at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1521)
        at java.base/java.lang.Thread.run(Thread.java:1570)
        at org.jboss.threads.JBossThread.run(JBossThread.java:483)
Caused by: jakarta.enterprise.inject.UnsatisfiedResolutionException: Unsatisfied dependency for type dev.langchain4j.store.embedding.inmemory.InMemoryEmbeddingStore and qualifiers [@Default]
        - injection target: com.emmanuelbernard.bearllm.Ingestor#store
        - declared on CLASS bean [types=[com.emmanuelbernard.bearllm.Ingestor, java.lang.Object], qualifiers=[@Default, @Any], target=com.emmanuelbernard.bearllm.Ingestor]
        at io.quarkus.arc.processor.Beans.resolveInjectionPoint(Beans.java:519)
        at io.quarkus.arc.processor.BeanInfo.init(BeanInfo.java:642)
        at io.quarkus.arc.processor.BeanDeployment.init(BeanDeployment.java:323)
        ... 12 more

        at io.quarkus.runner.bootstrap.AugmentActionImpl.runAugment(AugmentActionImpl.java:331)
        at io.quarkus.runner.bootstrap.AugmentActionImpl.createInitialRuntimeApplication(AugmentActionImpl.java:252)
        at io.quarkus.runner.bootstrap.AugmentActionImpl.createInitialRuntimeApplication(AugmentActionImpl.java:58)
        at io.quarkus.deployment.dev.IsolatedDevModeMain.firstStart(IsolatedDevModeMain.java:113)
        at io.quarkus.deployment.dev.IsolatedDevModeMain.accept(IsolatedDevModeMain.java:435)
        at io.quarkus.deployment.dev.IsolatedDevModeMain.accept(IsolatedDevModeMain.java:56)
        at io.quarkus.bootstrap.app.CuratedApplication.runInCl(CuratedApplication.java:138)
        at io.quarkus.bootstrap.app.CuratedApplication.runInAugmentClassLoader(CuratedApplication.java:93)
        at io.quarkus.deployment.dev.DevModeMain.start(DevModeMain.java:131)
        at io.quarkus.deployment.dev.DevModeMain.main(DevModeMain.java:62)
Caused by: io.quarkus.builder.BuildException: Build failure: Build failed due to errors
        [error]: Build step io.quarkus.arc.deployment.ArcProcessor#validate threw an exception: jakarta.enterprise.inject.spi.DeploymentException: jakarta.enterprise.inject.UnsatisfiedResolutionException: Unsatisfied dependency for type dev.langchain4j.store.embedding.inmemory.InMemoryEmbeddingStore and qualifiers [@Default]
        - injection target: com.emmanuelbernard.bearllm.Ingestor#store
        - declared on CLASS bean [types=[com.emmanuelbernard.bearllm.Ingestor, java.lang.Object], qualifiers=[@Default, @Any], target=com.emmanuelbernard.bearllm.Ingestor]
        at io.quarkus.arc.processor.BeanDeployment.processErrors(BeanDeployment.java:1546)
        at io.quarkus.arc.processor.BeanDeployment.init(BeanDeployment.java:338)
        at io.quarkus.arc.processor.BeanProcessor.initialize(BeanProcessor.java:167)
        at io.quarkus.arc.deployment.ArcProcessor.validate(ArcProcessor.java:490)
        at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
        at java.base/java.lang.reflect.Method.invoke(Method.java:580)
        at io.quarkus.deployment.ExtensionLoader$3.execute(ExtensionLoader.java:849)
        at io.quarkus.builder.BuildContext.run(BuildContext.java:256)
        at org.jboss.threads.ContextHandler$1.runWith(ContextHandler.java:18)
        at org.jboss.threads.EnhancedQueueExecutor$Task.doRunWith(EnhancedQueueExecutor.java:2516)
        at org.jboss.threads.EnhancedQueueExecutor$Task.run(EnhancedQueueExecutor.java:2495)
        at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1521)
        at java.base/java.lang.Thread.run(Thread.java:1570)
        at org.jboss.threads.JBossThread.run(JBossThread.java:483)
Caused by: jakarta.enterprise.inject.UnsatisfiedResolutionException: Unsatisfied dependency for type dev.langchain4j.store.embedding.inmemory.InMemoryEmbeddingStore and qualifiers [@Default]
        - injection target: com.emmanuelbernard.bearllm.Ingestor#store
        - declared on CLASS bean [types=[com.emmanuelbernard.bearllm.Ingestor, java.lang.Object], qualifiers=[@Default, @Any], target=com.emmanuelbernard.bearllm.Ingestor]
        at io.quarkus.arc.processor.Beans.resolveInjectionPoint(Beans.java:519)
        at io.quarkus.arc.processor.BeanInfo.init(BeanInfo.java:642)
        at io.quarkus.arc.processor.BeanDeployment.init(BeanDeployment.java:323)
        ... 12 more

        at io.quarkus.builder.Execution.run(Execution.java:123)
        at io.quarkus.builder.BuildExecutionBuilder.execute(BuildExecutionBuilder.java:79)
        at io.quarkus.deployment.QuarkusAugmentor.run(QuarkusAugmentor.java:160)
        at io.quarkus.runner.bootstrap.AugmentActionImpl.runAugment(AugmentActionImpl.java:327)
        ... 9 more
Caused by: jakarta.enterprise.inject.spi.DeploymentException: jakarta.enterprise.inject.UnsatisfiedResolutionException: Unsatisfied dependency for type dev.langchain4j.store.embedding.inmemory.InMemoryEmbeddingStore and qualifiers [@Default]
        - injection target: com.emmanuelbernard.bearllm.Ingestor#store
        - declared on CLASS bean [types=[com.emmanuelbernard.bearllm.Ingestor, java.lang.Object], qualifiers=[@Default, @Any], target=com.emmanuelbernard.bearllm.Ingestor]
        at io.quarkus.arc.processor.BeanDeployment.processErrors(BeanDeployment.java:1546)
        at io.quarkus.arc.processor.BeanDeployment.init(BeanDeployment.java:338)
        at io.quarkus.arc.processor.BeanProcessor.initialize(BeanProcessor.java:167)
        at io.quarkus.arc.deployment.ArcProcessor.validate(ArcProcessor.java:490)
        at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
        at java.base/java.lang.reflect.Method.invoke(Method.java:580)
        at io.quarkus.deployment.ExtensionLoader$3.execute(ExtensionLoader.java:849)
        at io.quarkus.builder.BuildContext.run(BuildContext.java:256)
        at org.jboss.threads.ContextHandler$1.runWith(ContextHandler.java:18)
        at org.jboss.threads.EnhancedQueueExecutor$Task.doRunWith(EnhancedQueueExecutor.java:2516)
        at org.jboss.threads.EnhancedQueueExecutor$Task.run(EnhancedQueueExecutor.java:2495)
        at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1521)
        at java.base/java.lang.Thread.run(Thread.java:1570)
        at org.jboss.threads.JBossThread.run(JBossThread.java:483)
Caused by: jakarta.enterprise.inject.UnsatisfiedResolutionException: Unsatisfied dependency for type dev.langchain4j.store.embedding.inmemory.InMemoryEmbeddingStore and qualifiers [@Default]
        - injection target: com.emmanuelbernard.bearllm.Ingestor#store
        - declared on CLASS bean [types=[com.emmanuelbernard.bearllm.Ingestor, java.lang.Object], qualifiers=[@Default, @Any], target                                                                                                                          =com.emmanuelbernard.bearllm.Ingestor]
        at io.quarkus.arc.processor.Beans.resolveInjectionPoint(Beans.java:519)
        at io.quarkus.arc.processor.BeanInfo.init(BeanInfo.java:642)
        at io.quarkus.arc.processor.BeanDeployment.init(BeanDeployment.java:323)
        ... 12 more

From

@ApplicationScoped
public class Ingestor {
    @Inject
    InMemoryEmbeddingStore store;

    @Inject
    EmbeddingModel embeddingModel;

    public void ingest(List<Document> documents) {
        EmbeddingStoreIngestor ingestor = EmbeddingStoreIngestor.builder()
                .embeddingStore(store)
                .embeddingModel(embeddingModel)
                //TODO adjust splitter to Markdown
                .documentSplitter(recursive(500, 0))
                .build();
        // Warning - this can take a long time...
        ingestor.ingest(documents);
    }
}

I did that because the Redis example seem to imply it works
https://docs.quarkiverse.io/quarkus-langchain4j/dev/retrievers.html

Adding this class "fixed" the problem

public class InMemoryEmbeddingStoreProvider {
    @Produces
    @ApplicationScoped
    public InMemoryEmbeddingStore create() {
        return new InMemoryEmbeddingStore<>();
    }
}

[geoand]

Can you try using InMemoryEmbeddingStore<TextSegment> as the type at the injection point?

Never mind the previous comment. I actually see we only configure InMemoryEmbeddingStore as a bean when quarkus-langchain4j-easy-rag is present.

@jmartisk does it make sense to move

quarkus-langchain4j/rag/easy-rag/deployment/src/main/java/io/quarkiverse/langchain4j/deployment/EasyRagProcessor.java

Line 54 in 82f6d29

public void createInMemoryEmbeddingStoreIfNoOtherExists(

into the core?

[jmartisk]

I'm not saying a strict no, but I feel that this is a bit too much magic to be enabled automatically in all cases.
And we might not be able to detect properly whether it is appropriate to do it - what if the application registers an embedding store (or multiple) by itself (then it won't produce a EmbeddingStoreBuildItem and it will be a bit tricky to detect it)?

[emmanuelbernard]

Yeah I found that some useful things are currently only in easy-rag:

1. the embedding store (what's its shortname to differentiate it form say the Redis store, what property to disambiguate when both are in the classpath. Unless we offer a quarkus-langchain4j-embeddingstore. But to be honest,m we want a configuration based way to swap stores, models etc so different profiles can use different versions.
2. the ApacheTikaDocumentParser, I'm not 100% sure but I suspect it can be useful outside easy-rag

Compounded to that, when you add easy rag to the classpath, you then have to set it up (path etc), so you cannot use it as a simple dependency. It makes the easy rag to less easy rag a big wall.

I wonder if we want to move these conversdations in Discussions as "Improve langchain4J integration". WDYT?

Emmanuel

[geoand]

I wonder if we want to move these conversdations in Discussions as "Improve langchain4J integration". WDYT?

Makes sense!

I'm on a phone now, so I'll check if I can move the issue when I get to a proper machine

[geoand]

Done

[emmanuelbernard]

I want to see how we can move forward.
At the moment, enabling embedding store provider is non intuitive.
I understand thjat the magic scares a bit @jmartisk
So what can we do to improve that experience.

My understanding is that at the moment we have the following behavior

• when a specific store dependency is there, we create a bean
• when easy rag is in, we create the embedded store bean
• there is not way to select the store we want to use if two or more dependencies are present

So I think we need to define the expected ideal behavior and a way to scale in complexity (e.g. multiple dependencies e.g. between dev and prod or two flows in a single app)

What do you think we should do?

[jmartisk]

Yeah, so the general case without Easy RAG:

• no embedding store dependency = no embedding store, you have to register something yourself
• one embedding store dependency = that one will get registered
• multiple embedding store dependencies = you get multiple embedding stores and you then distinguish between them using CDI by injecting the specific implementation class

With Easy RAG:

• no embedding store dependency = an in-memory store is created for the Easy RAG purposes
• one embedding store dependency = that one will be registered & used by Easy RAG
• multiple embedding store dependencies = all will get registered, but that breaks Easy RAG, because it doesn't know which store to use

The debatable points IMO are:

• Without Easy Rag, and no dependency
  • Here we could potentially create an in-memory store automatically. But we would have to make sure that
    • That the app is actually using some sort of RAG, otherwise it's useless
    • That the app isn't using/registering an embedding store in some custom way that Quarkus doesn't know about, and this is hard
• With Easy RAG, with multiple dependencies
  • We could provide a way to specify which embedding store should be used by the Easy RAG's ingestor and retrieval augmentor. Whether that's useful is debatable, Easy RAG was meant to be used in simple cases (quickstart-style), and having multiple stores doesn't sound very simple to me.

So what would you like to change here?

So I think we need to define the expected ideal behavior and a way to scale in complexity (e.g. multiple dependencies e.g. between dev and prod or two flows in a single app)

Just to make sure we're on the same page, this has nothing to do with dev mode, Easy RAG behaves the same in dev and prod