Add Dropbox Document Loader #7301
Conversation
…n-ath/langchainjs into add-dropbox-document-loader
dosubot
bot
added
the
size:XL
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎
1 Skipped Deployment
|
dosubot
bot
added
auto:documentation
|
Hi @jacoblee93, Can you take a look at this PR? I'd appreciate your feedback. |
| accessToken: "your-dropbox-access-token", | ||
| }, | ||
| unstructuredOptions: { | ||
| apiUrl: "http://localhost:8000/general/v0/general", // Replace with your Unstructured API URL |
There was a problem hiding this comment.
Let's emphasize somewhere that this wraps Unstructured
Should we call this DropboxUnstructuredLoader instead?
There was a problem hiding this comment.
Yes, we can rename the loader class to DropboxUnstructuredLoader
I want to confirm if I need to rename the file to say dropbox_unstructured.ts as well?
Also, I noticed that a few preexisting loaders utilize unstructured as well. Would they need to be renamed as well in the future?:
| console.log(docs[0].pageContent); | ||
| ``` | ||
|
|
||
| ## Configuration Options |
There was a problem hiding this comment.
Just link to API refs instead
| /** | ||
| * The file system module to use. Defaults to Node's `fs` module. | ||
| */ | ||
| fs?: typeof fsDefault; |
There was a problem hiding this comment.
I was thinking that a consumer might want to pass their own instance of fs to control where downloaded assets are stored in the interim before being sent out to Unstructured.
However by 7027578 the fs option is redundant and has been removed.
|
|
||
| await fsPromises.mkdir(path.dirname(localFilePath), { recursive: true }); | ||
|
|
||
| await fsPromises.writeFile(localFilePath, fileBinary, { |
There was a problem hiding this comment.
Would be nice to not need to use the file system
There was a problem hiding this comment.
Updated to not use the file system 7027578
This PR introduces support for the Dropbox Document Loader.
This is our first contribution to Langchain. I'd appreciate any feedback and suggestions for this PR.
Fixes #7031