Detector Status and Set Assignment Updates #499
Conversation
…ainline thresholds and all other station types to the ramp threshold values. Also accounted for manual threshold set values captured in the station_log model that contain user selected changes to the threshold set assignments. In the detector status model, we are now comparing counted values for various diagnostic tests against actual sample counts.
| when UPPER(dt_set_id) like 'LOW%' then 'Low_Volume' | ||
| when UPPER(dt_set_id) like 'RURAL%' then 'Rural' | ||
| when UPPER(dt_set_id) like 'URBAN_D11%' then 'Urban_D11' | ||
| when UPPER(dt_set_id) like 'D6_RAMPS%' then 'D6_Ramps' |
There was a problem hiding this comment.
I'm trying to understand the logic. Does this imply each detector has different thresholds for each category?
There was a problem hiding this comment.
The threshold set and method is applied at the station level. @thehanggit there isn't much documentation in the user guide but what I've been able to discover is that each station is associated with a threshold set based on the metadata coming from the configuration files sent by district or the threshold set can be manually defined in the PeMS application. The threshold set corresponds to the diagnostic criteria found in the diagnostic_threshold_values.csv seed file. If a threshold set value is not provided by district in the configuration file or in the PeMS application, the default is "Urban". The diagnostic test method (second column in the seed file) is based on the station type. Once we know the threshold set and method, we can apply the corresponding diagnostic criteria to stations and their associated detectors.
There was a problem hiding this comment.
The threshold set and method is applied at the station level. @thehanggit there isn't much documentation in the user guide but what I've been able to discover is that each station is associated with a threshold set based on the metadata coming from the configuration files sent by district or the threshold set can be manually defined in the PeMS application. The threshold set corresponds to the diagnostic criteria found in the
diagnostic_threshold_values.csvseed file. If a threshold set value is not provided by district in the configuration file or in the PeMS application, the default is "Urban". The diagnostic test method (second column in the seed file) is based on the station type. Once we know the threshold set and method, we can apply the corresponding diagnostic criteria to stations and their associated detectors.
Thanks! That means the detector status would be aligned with the old PeMS by implementing the similar threshold set.
There was a problem hiding this comment.
The threshold set and method is applied at the station level. @thehanggit there isn't much documentation in the user guide but what I've been able to discover is that each station is associated with a threshold set based on the metadata coming from the configuration files sent by district or the threshold set can be manually defined in the PeMS application. The threshold set corresponds to the diagnostic criteria found in the
diagnostic_threshold_values.csvseed file. If a threshold set value is not provided by district in the configuration file or in the PeMS application, the default is "Urban". The diagnostic test method (second column in the seed file) is based on the station type. Once we know the threshold set and method, we can apply the corresponding diagnostic criteria to stations and their associated detectors.Thanks! That means the detector status would be aligned with the old PeMS by implementing the similar threshold set.
As long as they are similar in the upper-level tables, we can cite their threshold as reference in yaml file to make it clear.
This PR updates the set assignment model to associate ML and HV stations to mainline threshold values and all other station types to the ramp threshold values. This also accounts for manual threshold set values captured in the station_log model that contain user selected changes to the threshold set assignments.
In the detector status model, we are now comparing counted values for various diagnostic tests against actual sample counts.
Based on my QC work, there are a few inconsistencies with the Constant value for some stations and we do not have the complete logic for the existing Feed Unstable status, but all other statuses appear consistent between the existing system and this project.
This fixes #398