pyuls

A small collection of tools(and one domain-specific language!) for pulling down the FCC's ULS databases and mooshing them into a SQLite database. Also does some indexing. And there's some bugs, specifically when there are pipe characters inside of field names and maybe one or two other places. And you'll need some disk space, the whole ULS is <32GB once indexed, but only just.

runme.sh will call the following five steps for you:

1. ./create_schema.sh path_to_database

• Creates empty tables in SQLite based on mysql format from FCC

2. ./fetch_databases.sh

• Erases 'databases' folder, prompts you for which databases you want, and downloads them to 'databases' folder

3. ./populate_databases.py path_to_database find databases -name *.dat

• Imports FCC ULS DAT files into SQLite DB

4. ./create_indexes.py path_to_database --index_file path_to_index_file

• Parse the index file and create the indexes it describes.
• Runme.sh defaults to example.index, which is also the language documentation.
• example.index creates indexes on columns I found useful, but may not be helpful to you. You'll likely want to create your own indexfile and define the columns you'll be searching or joining on.
• Optional - but without this most queries will be quite slow.

5. ./optimize.sh path_to_database

• Frees up any internal space in database
• Optional - rewrites entire database and is quite slow.