Monotonically Increasing Indices Assertions

[esanchez01]

In the Develop2 branch, there appears to be several assert statements across shadow_pricing.py and location_choice.py which check that the indices of a data frame are monotonically increasing. I ran the multi processed simple 3 zone example but passed in SANDAG inputs (inputs transformed to match example inputs) and failed these assertions. I included a stack trace below.

As a temporary workaround, I sorted the indices before the assertions were run. For example:

if not size_terms.index.is_monotonic_increasing:
    size_terms.sort_index(inplace=True)
assert size_terms.index.is_monotonic_increasing

Once the above change was made across all assertions, the entire model ran successfully.

With that said, I'm not quite sure if the indices being out of order is input or code related. Some of the assertions are ran directly after reading a table with inject.get_table() and then converting it into a data frame so it appears as if the data frame conversion does not guarantee sorted indices.

Traceback

mp_tasks - mp_models_0 - AssertionError exception running school_location model: --- mp_tasks - mp_models_0 - AssertionError exception caught in mp_run_simulation: Traceback (most recent call last): File "c:\users\esanc\.conda\envs\asimtest_multi_develop2\lib\site-packages\activitysim\core\mp_tasks.py", line 839, in mp_run_simulation run_simulation(queue, step_info, resume_after, shared_data_buffer) File "c:\users\esanc\.conda\envs\asimtest_multi_develop2\lib\site-packages\activitysim\core\mp_tasks.py", line 797, in run_simulation raise e File "c:\users\esanc\.conda\envs\asimtest_multi_develop2\lib\site-packages\activitysim\core\mp_tasks.py", line 794, in run_simulation pipeline.run_model(model) File "c:\users\esanc\.conda\envs\asimtest_multi_develop2\lib\site-packages\activitysim\core\pipeline.py", line 468, in run_model orca.run([step_name]) File "c:\users\esanc\.conda\envs\asimtest_multi_develop2\lib\site-packages\activitysim\core\orca.py", line 2033, in run step() File "c:\users\esanc\.conda\envs\asimtest_multi_develop2\lib\site-packages\activitysim\core\orca.py", line 842, in __call__ return self._func(**kwargs) File "c:\users\esanc\.conda\envs\asimtest_multi_develop2\lib\site-packages\activitysim\abm\models\location_choice.py", line 904, in school_location chunk_size, trace_hh_id, locutor, trace_label File "c:\users\esanc\.conda\envs\asimtest_multi_develop2\lib\site-packages\activitysim\abm\models\location_choice.py", line 763, in iterate_location_choice trace_label=tracing.extend_trace_label(trace_label, 'i%s' % iteration)) File "c:\users\esanc\.conda\envs\asimtest_multi_develop2\lib\site-packages\activitysim\abm\models\location_choice.py", line 575, in run_location_choice dest_size_terms = shadow_price_calculator.dest_size_terms(segment_name) File "c:\users\esanc\.conda\envs\asimtest_multi_develop2\lib\site-packages\activitysim\abm\tables\shadow_pricing.py", line 521, in dest_size_terms assert size_terms.index.is_monotonic_increasing

[toliwaga]

Yes, activitysim is assuming the input land_use table is sorted by zone id. This should not be required.

shadow pricing wants the the size_term table to be in a known order since the data in it is being shared across sub-processes as numpy array. But there is no reason this should emerged at the input file level.

fixing this right now.

[toliwaga]

fixed now in develop2

[esanchez01]

I pulled the latest changes from develop and I am still encountering a similar issue. I included a stack trace below. I will apply the above fix for now.

Let me know if any additional information is needed.

Traceback

18/03/2021 05:46:28 - WARNING - activitysim.core.mp_tasks - AssertionError exception running school_location model: school_destination_size not is_monotonic_increasing 18/03/2021 05:46:28 - ERROR - activitysim.core.mp_tasks - mp_tasks - mp_models_0 - AssertionError exception caught in mp_run_simulation: school_destination_size not is_monotonic_increasing 18/03/2021 05:46:28 - ERROR - activitysim.core.mp_tasks - --- Traceback (most recent call last): File "c:\users\esanc\.conda\envs\asimtest_multi_develop\lib\site-packages\activitysim\core\mp_tasks.py", line 840, in mp_run_simulation run_simulation(queue, step_info, resume_after, shared_data_buffer) File "c:\users\esanc\.conda\envs\asimtest_multi_develop\lib\site-packages\activitysim\core\mp_tasks.py", line 797, in run_simulation raise e File "c:\users\esanc\.conda\envs\asimtest_multi_develop\lib\site-packages\activitysim\core\mp_tasks.py", line 794, in run_simulation pipeline.run_model(model) File "c:\users\esanc\.conda\envs\asimtest_multi_develop\lib\site-packages\activitysim\core\pipeline.py", line 468, in run_model orca.run([step_name]) File "c:\users\esanc\.conda\envs\asimtest_multi_develop\lib\site-packages\activitysim\core\orca.py", line 2033, in run step() File "c:\users\esanc\.conda\envs\asimtest_multi_develop\lib\site-packages\activitysim\core\orca.py", line 842, in __call__ return self._func(**kwargs) File "c:\users\esanc\.conda\envs\asimtest_multi_develop\lib\site-packages\activitysim\abm\models\location_choice.py", line 904, in school_location chunk_size, trace_hh_id, locutor, trace_label File "c:\users\esanc\.conda\envs\asimtest_multi_develop\lib\site-packages\activitysim\abm\models\location_choice.py", line 742, in iterate_location_choice spc = shadow_pricing.load_shadow_price_calculator(model_settings) File "c:\users\esanc\.conda\envs\asimtest_multi_develop\lib\site-packages\activitysim\abm\tables\shadow_pricing.py", line 709, in load_shadow_price_calculator num_processes, data, lock) File "c:\users\esanc\.conda\envs\asimtest_multi_develop\lib\site-packages\activitysim\abm\tables\shadow_pricing.py", line 129, in __init__ f"{size_table_name(self.model_selector)} not is_monotonic_increasing" AssertionError: school_destination_size not is_monotonic_increasing