Skip to content

@bitextor Bitextor Team

Change the repository type filter

All

    Repositories list

    29 repositories

    • monocleaner

      Public
      Python
      •
      GNU General Public License v3.0
      •1•7•1•0•Updated Dec 2, 2024Dec 2, 2024

    • bicleaner-hardrules

      Public
      Pre-filtering step for bicleaner
      Python
      •
      GNU General Public License v3.0
      •2•4•0•0•Updated Dec 2, 2024Dec 2, 2024

    • biroamer

      Public
      Utility that will help you to ROAM (Random Omit Anonymize and Mix) your parallel corpus.
      Python
      •
      GNU General Public License v3.0
      •2•10•0•1•Updated Nov 25, 2024Nov 25, 2024

    • scrawl

      Public
      Playwright-based web crawler
      Python
      •
      GNU General Public License v3.0
      •0•0•0•0•Updated Nov 14, 2024Nov 14, 2024

    • bitextor

      Public
      Bitextor generates translation memories from multilingual websites
      crawlerdictionariestokenizermachine-translationwgetapertiumneural-machine-translationwarctmxstatistical-machine-translation
      Python
      •
      GNU General Public License v3.0
      •43•292•3•4•Updated Nov 11, 2024Nov 11, 2024

    • pdf-extract

      Public
      PDF parser and converter to HTML
      Java
      •
      GNU General Public License v3.0
      •14•84•4•1•Updated Oct 3, 2024Oct 3, 2024

    • bifixer

      Public
      Tool to fix bitexts and tag near-duplicates for removal
      Python
      •
      GNU General Public License v3.0
      •3•29•0•0•Updated Aug 19, 2024Aug 19, 2024

    • warc2text

      Public
      Extracts plain text, language identification and more metadata from WARC records
      C++
      •
      MIT License
      •5•20•12•3•Updated Aug 8, 2024Aug 8, 2024

    • bicleaner-ai

      Public
      Bicleaner fork that uses neural networks
      Python
      •
      GNU General Public License v3.0
      •4•38•1•0•Updated Jul 26, 2024Jul 26, 2024

    • bicleaner

      Public
      Bicleaner is a parallel corpus classifier/cleaner that aims at detecting noisy sentence pairs in a parallel corpus.
      Python
      •
      GNU General Public License v3.0
      •22•152•0•1•Updated Jun 18, 2024Jun 18, 2024

    • monotextor

      Public
      Python
      •
      GNU General Public License v3.0
      •1•6•0•0•Updated May 31, 2023May 31, 2023

    • bitextor-testing-output

      Public
      Repository for storing testing outputs from Bitextor
      GNU General Public License v3.0
      •0•0•0•0•Updated May 29, 2023May 29, 2023

    • prevertical2text

      Public
      Extracts plain text, language identification and more metadata from Spiderling prevertical files
      C++
      •
      MIT License
      •0•2•0•0•Updated May 17, 2023May 17, 2023

    • fastText

      Public
      Library for fast text representation and classification.
      HTML
      •
      MIT License
      •4.7k•0•0•0•Updated May 4, 2023May 4, 2023

    • deferred-crawling

      Public
      Reconstructs sentences using deferred crawling standoff annotations from Bitextor
      Python
      •
      MIT License
      •0•0•0•0•Updated May 4, 2023May 4, 2023

    • bicleaner-ai-data

      Public
      Repository of Bicleaner AI models
      Other
      •0•5•0•0•Updated Mar 28, 2023Mar 28, 2023

    • bleualign-cpp

      Public
      C++
      •
      GNU General Public License v3.0
      •2•7•2•0•Updated Mar 10, 2023Mar 10, 2023

    • bicleaner-data

      Public
      Repository for data models, dictionaries and more resources for Bicleaner
      GNU General Public License v3.0
      •0•6•0•0•Updated Dec 15, 2022Dec 15, 2022

    • vecalign

      Public
      Improved Sentence Alignment in Linear Time and Space
      Python
      •
      Apache License 2.0
      •30•2•0•0•Updated Dec 4, 2022Dec 4, 2022

    • python-apachetika

      Public
      Python interface to Apache Tika, HTML extraction from PDF
      Python
      •
      Other
      •143•0•0•0•Updated Nov 30, 2022Nov 30, 2022

    • loomchild-segment-py

      Public
      Python module to interface with Java Loomchild sentence segmenter
      Python
      •
      GNU General Public License v3.0
      •1•1•1•0•Updated Nov 28, 2022Nov 28, 2022

    • bicleaner-ai-glove

      Public
      Fork of glove-python to distribute binary builds
      Python
      •
      Apache License 2.0
      •320•0•0•0•Updated Aug 12, 2022Aug 12, 2022

    • neural-document-aligner

      Public
      Document aligner which uses neural technologies to search matches across bilingual documents
      Python
      •
      GNU General Public License v3.0
      •3•7•0•0•Updated Jun 9, 2022Jun 9, 2022

    • bitextor-neural

      Public archive
      Bitextor Neural generates translation memories from multilingual websites using state-of-the-art Machine Learning tools
      Python
      •
      GNU General Public License v3.0
      •0•3•0•0•Updated Jun 3, 2022Jun 3, 2022

    • monocleaner-data

      Public
      Monocleaner models repository
      GNU General Public License v3.0
      •0•1•0•0•Updated Nov 18, 2021Nov 18, 2021

    • hunalign

      Public
      Sentence aligner
      C++
      •
      GNU Lesser General Public License v3.0
      •38•0•0•0•Updated May 21, 2021May 21, 2021

    • cld2

      Public
      Compact Language Detector 2
      C++
      •
      Apache License 2.0
      •131•0•0•0•Updated May 4, 2021May 4, 2021

    • python-pdfextract

      Public
      Python interface to pdf-extract, HTML extraction from PDF
      Python
      •
      Other
      •143•6•0•0•Updated Sep 3, 2020Sep 3, 2020

    • bitextor-data

      Public
      Repository for data models, dictionaries and more resources for Bitextor
      GNU General Public License v3.0
      •0•5•0•0•Updated Feb 7, 2020Feb 7, 2020