-
Notifications
You must be signed in to change notification settings - Fork 711
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
540 Release Candidate #14247
540 Release Candidate #14247
Commits on May 14, 2024
-
Configuration menu - View commit details
-
Copy full SHA for deb0a45 - Browse repository at this point
Copy the full SHA deb0a45View commit details -
Configuration menu - View commit details
-
Copy full SHA for b3d491b - Browse repository at this point
Copy the full SHA b3d491bView commit details -
Configuration menu - View commit details
-
Copy full SHA for d8a42c0 - Browse repository at this point
Copy the full SHA d8a42c0View commit details -
Configuration menu - View commit details
-
Copy full SHA for 5967577 - Browse repository at this point
Copy the full SHA 5967577View commit details
Commits on May 21, 2024
-
Configuration menu - View commit details
-
Copy full SHA for fcd4e9c - Browse repository at this point
Copy the full SHA fcd4e9cView commit details -
[SPARKNLP-1037] Adding addFile changes to to replace broadcast in all…
… ONNX based annotators (#14236) * [SPARKNLP-1011] Adding changes to transfer ONNX files on executors through Spark files feature * [SPARKNLP-1011] Adding missing copyright comment * [SPARKNLP-1011] Adding changes to add prefix for models with onnx_data file * [SPARKNLP-1037] Adding changes to transfer ONNX files on executors via addFile * [SPARKNLP-1037] Adding unique suffix to avoid duplication in spark files
Configuration menu - View commit details
-
Copy full SHA for 4419a70 - Browse repository at this point
Copy the full SHA 4419a70View commit details -
Integrating OpenVINO Runtime in Spark NLP (#14200)
* Use OpenVINO model engine for BertEmbeddings Add OpenVINO model engine wrapper Add default buffer size for reading weights file Read OpenVINO IR format models * Use Long Tensors with XlmRoberta * Add OpenVINO support for RoBerta and XlmRoBerta embeddings * Fix data type and formatting * Add OpenVINO BERT test - Resole merge conflicts - Add test for openvino load model * Update Python APIs to use OpenVINO * Add param to enable OpenVINO through Python API * Formatting changes * Add OpenVINO support for E5 Embeddings * Enable OpenVINO backend for E5 Embeddings * Update Python APIs * Resolve merge issues * Add OpenVINO support for T5 * Read and write encoder-decoder models with OpenVINO * OpenVINO Async Inference * Refactor and cleanup * Update comments * Add config to set OpenVINO inference device * Add OpenVINO support for BERT Sentence Embeddings * Formatting * Openvino synchronous inference * Refactoring: OV Model Conversion * BertSentenceEmbeddings Python API * Enable OpenVINO support for Llama2 * Read/write Llama2 Transformer with OpenVINO * Bugfix: Update saved model filename * Replace broadcast with addFile for OpenVINO-based annotators * Add OpenVINO Wrapper tests * Add suffix to avoid duplication in Spark Files * Set default OV inference device to CPU * Use CPU by default for OpenVINO inference due to error loading device config in cluster envs * Bugfix: Read serialized model from folder
Configuration menu - View commit details
-
Copy full SHA for fabc4ab - Browse repository at this point
Copy the full SHA fabc4abView commit details
Commits on May 24, 2024
-
Fixing colab notebook bugs (#14249)
* adding code for spp files and updating notebooks * fixed code and notebook for xlnet embeddings * fixing more notebooks * adding notebook for bert sentence embeddings
Configuration menu - View commit details
-
Copy full SHA for 9430402 - Browse repository at this point
Copy the full SHA 9430402View commit details -
Configuration menu - View commit details
-
Copy full SHA for 3f59375 - Browse repository at this point
Copy the full SHA 3f59375View commit details -
Configuration menu - View commit details
-
Copy full SHA for d083420 - Browse repository at this point
Copy the full SHA d083420View commit details -
Sparknlp 1035 test all notebooks to import tensor flow models to spar…
…k nlp (#14238) * Updating file version in XlmRobertaForZeroShotClassification * updating transformer versions
Configuration menu - View commit details
-
Copy full SHA for e0e28e8 - Browse repository at this point
Copy the full SHA e0e28e8View commit details -
Adding caching to streamlit demos (#14232)
* Delete examples/demos/streamlit/CLIPForZeroShotClassification/inputs directory * Add files via upload * Update streamlit_app.py * Update streamlit_app.py * Update streamlit_app.py * Update streamlit_app.py * Update streamlit_app.py * Update streamlit_app.py * Update streamlit_app.py * Update streamlit_app.py * Update streamlit_app.py * Update streamlit_app.py * Update streamlit_app.py * Update streamlit_app.py * Update streamlit_app.py * Update streamlit_app.py * Update streamlit_app.py * Update streamlit_app.py * Update streamlit_app.py * Update streamlit_app.py * Update streamlit_app.py * Update streamlit_app.py * Update streamlit_app.py * Update streamlit_app.py * Update streamlit_app.py * Update streamlit_app.py * Update streamlit_app.py * Update streamlit_app.py * Update streamlit_app.py * Update streamlit_app.py * Update streamlit_app.py * Update streamlit_app.py * Update streamlit_app.py * Update streamlit_app.py * Update streamlit_app.py * Update streamlit_app.py * Update streamlit_app.py * Update streamlit_app.py * Update streamlit_app.py * Update streamlit_app.py * Update streamlit_app.py * Delete examples/demos/streamlit/run_demos.ipynb * Add files via upload
Configuration menu - View commit details
-
Copy full SHA for 262b802 - Browse repository at this point
Copy the full SHA 262b802View commit details -
Configuration menu - View commit details
-
Copy full SHA for 74f5151 - Browse repository at this point
Copy the full SHA 74f5151View commit details
Commits on Jun 3, 2024
-
Configuration menu - View commit details
-
Copy full SHA for c2048be - Browse repository at this point
Copy the full SHA c2048beView commit details -
Configuration menu - View commit details
-
Copy full SHA for adc193e - Browse repository at this point
Copy the full SHA adc193eView commit details
Commits on Jun 5, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 7274281 - Browse repository at this point
Copy the full SHA 7274281View commit details
Commits on Jun 10, 2024
-
Sparknlp 1016 implement mp net for token classification (#14322)
* implementing mpnet for tokenclassification * changing default pretrained model name --------- Co-authored-by: Maziyar Panahi <maziyar.panahi@iscpif.fr>
Configuration menu - View commit details
-
Copy full SHA for 9c075f8 - Browse repository at this point
Copy the full SHA 9c075f8View commit details -
Configuration menu - View commit details
-
Copy full SHA for 0ea5898 - Browse repository at this point
Copy the full SHA 0ea5898View commit details -
SparkNLP - 995 Introducing MistralAI LLMs (#14318)
* added mistral * Mistral python API
Configuration menu - View commit details
-
Copy full SHA for 4583ccf - Browse repository at this point
Copy the full SHA 4583ccfView commit details -
SparkNLP 1043 integrate new casual lm annotators to use open vino (#1…
…4319) * Phi2 scala api * Phi2 python api * Phi2 python and scala tests * Phi2 python and scala tests * added M2M100 openvino implementation * added phi2 openvino implementation * added openvino flag to python --------- Co-authored-by: Maziyar Panahi <maziyar.panahi@iscpif.fr>
Configuration menu - View commit details
-
Copy full SHA for cdb031a - Browse repository at this point
Copy the full SHA cdb031aView commit details -
Fixed LLAMA generation bug (#14320)
* fixed LLAMA generation bug * update params
Configuration menu - View commit details
-
Copy full SHA for 3054d4c - Browse repository at this point
Copy the full SHA 3054d4cView commit details -
Configuration menu - View commit details
-
Copy full SHA for b4000d3 - Browse repository at this point
Copy the full SHA b4000d3View commit details -
Configuration menu - View commit details
-
Copy full SHA for 85c90dd - Browse repository at this point
Copy the full SHA 85c90ddView commit details
Commits on Jun 12, 2024
-
Add Pooling Average to Broken XXXForSentenceEmbedding annotators (#14328
) * SPARKNLP-1036: Onnx Example notebooks (#14234) * SPARKNLP-1036: Fix dev python kernel names * SPARKNLP-1036: Bump transformers version * SPARKNLP-1036: Fix Colab buttons * SPARKNLP-1036: Pin onnx version for compatibility * SPARKNLP-1036: Upgrade Spark version * SPARKNLP-1036: Minor Fixes * SPARKNLP-1036: Clean Metadata * SPARKNLP-1036: Add/Adjust Documentation - Note for supported Spark Version of Annotators - added missing Documentation for BGEEmbeddings * Fixies (#14307) * adding fix for broken annotators --------- Co-authored-by: Devin Ha <33089471+DevinTDHa@users.noreply.github.com> Co-authored-by: Lev <agsfer@gmail.com> Co-authored-by: Maziyar Panahi <maziyar.panahi@iscpif.fr>
Configuration menu - View commit details
-
Copy full SHA for 1cba7e3 - Browse repository at this point
Copy the full SHA 1cba7e3View commit details -
Configuration menu - View commit details
-
Copy full SHA for 903e780 - Browse repository at this point
Copy the full SHA 903e780View commit details -
Configuration menu - View commit details
-
Copy full SHA for 54027a4 - Browse repository at this point
Copy the full SHA 54027a4View commit details
Commits on Jun 21, 2024
-
Fix models link on FAQ (#14333)
* SPARKNLP-1036: Onnx Example notebooks (#14234) * SPARKNLP-1036: Fix dev python kernel names * SPARKNLP-1036: Bump transformers version * SPARKNLP-1036: Fix Colab buttons * SPARKNLP-1036: Pin onnx version for compatibility * SPARKNLP-1036: Upgrade Spark version * SPARKNLP-1036: Minor Fixes * SPARKNLP-1036: Clean Metadata * SPARKNLP-1036: Add/Adjust Documentation - Note for supported Spark Version of Annotators - added missing Documentation for BGEEmbeddings * Fixies (#14307) * Fix models link --------- Co-authored-by: Devin Ha <33089471+DevinTDHa@users.noreply.github.com> Co-authored-by: Lev <agsfer@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for 4356794 - Browse repository at this point
Copy the full SHA 4356794View commit details -
adding onnx support and average pooling (#14330)
* adding onnx support and average pooling * Delete examples/python/transformers/HuggingFace_in_Spark_NLP_RoBertaSentenceEmbeddings.ipynb.txt * adding openvino
Configuration menu - View commit details
-
Copy full SHA for 5a86b70 - Browse repository at this point
Copy the full SHA 5a86b70View commit details -
uploading UAEEmbeddings notebook (#14324)
* SPARKNLP-1036: Onnx Example notebooks (#14234) * SPARKNLP-1036: Fix dev python kernel names * SPARKNLP-1036: Bump transformers version * SPARKNLP-1036: Fix Colab buttons * SPARKNLP-1036: Pin onnx version for compatibility * SPARKNLP-1036: Upgrade Spark version * SPARKNLP-1036: Minor Fixes * SPARKNLP-1036: Clean Metadata * SPARKNLP-1036: Add/Adjust Documentation - Note for supported Spark Version of Annotators - added missing Documentation for BGEEmbeddings * Fixies (#14307) * uploading UAEEmbeddings notebook Adding UAEEmbeddings example notebook --------- Co-authored-by: Devin Ha <33089471+DevinTDHa@users.noreply.github.com> Co-authored-by: Lev <agsfer@gmail.com> Co-authored-by: Maziyar Panahi <maziyar.panahi@iscpif.fr>
Configuration menu - View commit details
-
Copy full SHA for ac9de09 - Browse repository at this point
Copy the full SHA ac9de09View commit details
Commits on Jun 26, 2024
-
Configuration menu - View commit details
-
Copy full SHA for e88682c - Browse repository at this point
Copy the full SHA e88682cView commit details
Commits on Jun 28, 2024
-
Refactor OpenAIEmbeddings (#14334)
* SPARKNLP-1036: Onnx Example notebooks (#14234) * SPARKNLP-1036: Fix dev python kernel names * SPARKNLP-1036: Bump transformers version * SPARKNLP-1036: Fix Colab buttons * SPARKNLP-1036: Pin onnx version for compatibility * SPARKNLP-1036: Upgrade Spark version * SPARKNLP-1036: Minor Fixes * SPARKNLP-1036: Clean Metadata * SPARKNLP-1036: Add/Adjust Documentation - Note for supported Spark Version of Annotators - added missing Documentation for BGEEmbeddings * Fixies (#14307) * refactor OpenAIEmbeddings in Scala * refactor OpenAIEmbeddings in Python * add pytest.mark.slow and improve doc --------- Co-authored-by: Devin Ha <33089471+DevinTDHa@users.noreply.github.com> Co-authored-by: Lev <agsfer@gmail.com> Co-authored-by: Maziyar Panahi <maziyar.panahi@iscpif.fr>
Configuration menu - View commit details
-
Copy full SHA for 09dc500 - Browse repository at this point
Copy the full SHA 09dc500View commit details -
Configuration menu - View commit details
-
Copy full SHA for 9d235e0 - Browse repository at this point
Copy the full SHA 9d235e0View commit details -
Configuration menu - View commit details
-
Copy full SHA for 036fc50 - Browse repository at this point
Copy the full SHA 036fc50View commit details -
Configuration menu - View commit details
-
Copy full SHA for 86e6725 - Browse repository at this point
Copy the full SHA 86e6725View commit details
Commits on Jun 29, 2024
-
Configuration menu - View commit details
-
Copy full SHA for a5b88ad - Browse repository at this point
Copy the full SHA a5b88adView commit details -
Configuration menu - View commit details
-
Copy full SHA for 1502757 - Browse repository at this point
Copy the full SHA 1502757View commit details -
Configuration menu - View commit details
-
Copy full SHA for 595b8f4 - Browse repository at this point
Copy the full SHA 595b8f4View commit details