Home
The app gives the possibility to perform a search into the Ukrainian government Open data portal datasets.
At this moment (November 2021) integrated Information on missing citizens,
Information about people hiding from the authorities,
Unified register of debtors,
Unified State Register of Legal Entities, Individual Entrepreneurs and Public Associations
and Integrated Unified State Register of Lustrated Persons.
To avoid overwhelming screen output the first 10 results are shown. Whole results datasets are saving into HTML files.
- 16Gb+ RAM
- SSD
v1.9 Release. 24/11/2021
- Integrated Unified State Register of Lustrated Persons
- Docstrings added
- Unit tests enhancements
- Refactoring and optimization
v1.8.5. Release. 20/11/2021
- Refactoring to implement PEP 8
v1.8. Release. 04/11/2021
- Performance optimization
- Refactoring
- Added unit tests
- Added check if a dataset is outdated
- Error handling added into Search functions
- Updated dependencies
v1.7. Release. 28/10/2021
- Dockerizing
- Switched to Python 3.10.0
- Added setup script for run in Docker
- Added the run script
- Added requirements.txt
- Big refactoring and code optimization
- Integrated Unified State Register of Legal Entities, Individual Entrepreneurs and Public Associations
- Full parsing XML (finally!)
- Save parsed XML into MongoDB as a set of JSON
- Clear the results' folder before the search
- Try to use Garbage Collection to decrease RAM consumption
- Fixed: the app was crashing when a search result can't fit RAM limitations due to by default aggregation in MongoDB occurs in memory and pipeline stages have a limit of 100 Mb RAM. Use "allowDiskUse"
- Integrated Unified register of debtors
- Get ZIP archive
- Unzip resources
- Read and parse huge CSV
- Convert huge CSV to JSON
- Purging temp files
- Added performance metric into the log file: requests execution time
- JSON downloading, parsing, and saving into MongoDB
- Full-text search into datasets
- Search results datasets are saving into HTML files
- Integrated the missing person register and the wanted person register.
- A full-text search could be performed in any of the three languages: Russian, Ukrainian, and English. Results are showing only in English.
- Manual refreshing datasets
- Errors handling
- Full logging
- Built-in managing database operations: creating/truncating collections, creating/deleting text indexes
- Showing datasets the last update date
To be provided soon
Developed in PyCharm - The Python IDE for Professional Developers.
License kindly provided by JetBrains Community Support Team
