Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

(re) draft Census connector #978

Merged
merged 14 commits into from
Feb 20, 2024
78 changes: 78 additions & 0 deletions docs/census.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
Census
=============

********
Overview
********

Connects to the Census API--it has been tested with the ACS and Economic Survey endpoints.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add a link to the Census API here? Also, this is super nitpicky, but do you mind editing this so there's a period instead of a double dash between the two sentences? Sorry to be such a grammar nerd 🤓

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks Shauna! I fixed these and the other stuff. For the index.html file, I just downloaded the original and replaced my banged-up version :).

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great! Going to merge now.


.. note::
Authentication
The API requires a key that you can get `here <https://www.census.gov/data/developers.html>`_ (click on the box that says "Request a KEY")



==========
Quickstart
==========

To instantiate the Census class, either store your ``CENSUS_API_KEY`` as an environment
variable or pass it as an argument:

.. code-block:: python

from parsons import Census

# First approach: Use API key environment variables

# In Mac OS, for example, set your environment variables like so:
# export CENSUS_API_KEY='MY_CENSUS_KEY'

census = ActBlue()

# Second approach: Pass API key as arguments
actblue = ActBlue(api_key = 'MY_CENSUS_KEY')



**Example 1**

.. code-block:: python

from parsons import Census
# these variables tell the API what data we want
shaunagm marked this conversation as resolved.
Show resolved Hide resolved
year = '2019'
dataset_acronym = '/acs/acs1'
variables = 'NAME,B01001_001E'
location = 'us:1'

# here's our connector
census = Census(api_key=acs_key)

# now pull data into a Parsons table
t = census.get_census(year,dataset_acronym,variables,location)
shaunagm marked this conversation as resolved.
Show resolved Hide resolved
print(t)

This example pulls the population of the US in 2019 from the ACS 1-year estimates.

**Example 2**

.. code-block:: python

year = '2017'
dataset_acronym = '/ecnbasic'
variables = 'NAICS2017_LABEL,NAICS2017,GEO_ID,FIRM,PAYANN'
location = 'state:51'
census = Census(api_key=acs_key)

t = census.get_census(year,dataset_acronym,variables,location)
shaunagm marked this conversation as resolved.
Show resolved Hide resolved
print(t)


This example pulls data on payroll by industry in Virginia from the 2017 Economic Census.

***API***

.. autoclass :: parsons.Census
:inherited-members:
2 changes: 1 addition & 1 deletion docs/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -3,4 +3,4 @@
<meta http-equiv="refresh" content="0; url=html/index.html" />
</head>
<body></body>
</html>
</html>
1 change: 1 addition & 0 deletions docs/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -193,6 +193,7 @@ Indices and tables
box
braintree
capitolcanary
census
civis
controlshift
copper
Expand Down
1 change: 1 addition & 0 deletions parsons/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,7 @@
("parsons.braintree.braintree", "Braintree"),
("parsons.capitol_canary.capitol_canary", "CapitolCanary"),
["parsons.catalist.catalist", "CatalistMatch"],
("parsons.census.census", "Census"),
("parsons.civis.civisclient", "CivisClient"),
("parsons.controlshift.controlshift", "Controlshift"),
("parsons.copper.copper", "Copper"),
Expand Down
Empty file added parsons/census/__init__.py
Empty file.
64 changes: 64 additions & 0 deletions parsons/census/census.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
import logging
from parsons.utilities.api_connector import APIConnector
from parsons.utilities import check_env
from parsons import Table

logger = logging.getLogger(__name__)


class Census(object):
"""
Class that creates a connector to the Census Bureau API
"""

def __init__(self, api_key=None):
"""
Instantiate Census class.

Args:
api_key: string, key for Census API access
(optional, can also be pulled from environment variable CENSUS_API_KEY)
"""
self.api_key = check_env.check("CENSUS_API_KEY", api_key)
self.host = "https://api.census.gov/data"

def get_census(
self, year=None, dataset_acronym=None, variables=None, location=None
):
"""
Pull census data. For background check out the Census API Guide at
https://www.census.gov/data/developers/guidance/api-user-guide.html
shaunagm marked this conversation as resolved.
Show resolved Hide resolved

Args:
year: 4-digit string or integer
e.g. '2019' or 2019

dataset_acronym: string with dataset name
e.g. '/acs/acs1'

variables: comma-separated string with variable names
e.g. 'NAME,B01001_001E'

location: string with desired locations
e.g. 'us:*'

Return:
Parsons table with data
"""
# set up the URL
g = "?get="
usr_key = f"&key={self.api_key}"
year = str(year) # in case someone passes int
location = "&for=" + location
query_url = (
f"{self.host}/{year}{dataset_acronym}{g}{variables}{location}{usr_key}"
)

# create connector
connector = APIConnector(uri=self.host)

# pull data
response = connector.get_request(url=query_url)

# turn into Parsons table and return it
return Table(response)
36 changes: 36 additions & 0 deletions test/test_census/test_census.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
import unittest
import requests_mock
from test.utils import mark_live_test
from parsons import Table, Census


class TestCensus(unittest.TestCase):
def setUp(self):
self.census = Census()

def tearDown(self):
pass

@mark_live_test
def test_get_census_live_test(self):
year = "2019"
dataset_acronym = "/acs/acs1"
variables = "NAME,B01001_001E"
location = "for=state:*"
table = self.census.get_census(year, dataset_acronym, variables, location)
self.assertEqual(len(table), 52)
self.assertEqual(table[0]["NAME"], "Illinois")
self.assertIsInstance(table, Table)

@requests_mock.Mocker()
def test_get_census_mock_test(self, m):
year = "2019"
dataset_acronym = "/acs/acs1"
variables = "NAME,B01001_001E"
location = "for=us:1"
test_json = {"NAME": "United States", "B01001_001E": "328239523", "us": "1"}
table = m.census.get_census(
year, dataset_acronym, variables, location, json=test_json
)
self.assertEqual(table[0]["B01001_001E"], "328239523")
self.assertEqual(table[0]["NAME"], "United States")
Loading