Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/2485 reimplement intersection strategy for sections and flows #416

Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
113 commits
Select commit Hold shift + click to select a range
affecc2
Add pygeos package
randy-seng Nov 9, 2023
bfd2cdc
Extend TrackDataset interface with methods to get tracks intersecting…
randy-seng Nov 10, 2023
67bff15
Store tracks in TrackDataset in separate storage as pygeos geometries
randy-seng Nov 10, 2023
ca19c6c
Merge branch 'main' into feature/2485-reimplement-intersection-strate…
randy-seng Nov 10, 2023
6c59153
Order saved track geometries by offset in track dataset
randy-seng Nov 13, 2023
3bd4595
Move group_sections_by_offset function
randy-seng Nov 13, 2023
945f690
Add getting intersection points from TrackDataset with sections
randy-seng Nov 14, 2023
04dc3cc
Add documentation
randy-seng Nov 14, 2023
3e08072
Extend TrackDataset with method to get boolean mask of track coordina…
randy-seng Nov 14, 2023
d3ae3a8
Implement whether tracks in TrackDataset intersect with given sections
randy-seng Nov 14, 2023
57acfc2
Add getting intersection points from PandasTrackDataset with sections
randy-seng Nov 14, 2023
db50e02
Add documentation
randy-seng Nov 14, 2023
bec63df
Extend TrackDataset with method to get boolean mask of track coordina…
randy-seng Nov 14, 2023
e3fd49c
Implement whether tracks in PandasTrackDataset intersect with given s…
randy-seng Nov 14, 2023
51d1438
Merge remote-tracking branch 'origin/feature/2485-reimplement-interse…
randy-seng Nov 14, 2023
01ca961
Refactor track geometry dataset in PandasTrackDataset to own module
randy-seng Nov 16, 2023
90e221d
Move constants to top
randy-seng Nov 16, 2023
f32604d
Add method to serialise PygeosTrackGeometryDataset to dictionary
randy-seng Nov 16, 2023
53ac4a0
Change way to create PygeosTrackGeometryDataset
randy-seng Nov 16, 2023
dc42018
Fix access to get track ids from track geometry DataFrame
randy-seng Nov 16, 2023
6578642
Extract method to create pygeos track objects from PyGeosTrackDataset…
randy-seng Nov 16, 2023
7e9081f
Disregard intersections and intersects column when getting dictionary…
randy-seng Nov 16, 2023
5ab8275
Ensure correct instantiation of PyGeosTrackGeometryDataset
randy-seng Nov 16, 2023
a071160
Use BASE_GEOMETRY as default offset when creating new track geometry …
randy-seng Nov 16, 2023
c5ffe12
Add property informing whether PygeosTrackGeometryDataset is empty
randy-seng Nov 16, 2023
5326f98
Add property to get track ids from tracks stored in PygeosTrackGeomet…
randy-seng Nov 16, 2023
876387d
Add missing docstring
randy-seng Nov 16, 2023
3e4cb49
Remove unused method
randy-seng Nov 16, 2023
b6579d3
Implement adding tracks to existing track geometry dataset
randy-seng Nov 16, 2023
3ae35d8
Add __init__ files
randy-seng Nov 17, 2023
b329cf9
Implement removing track geometries from dataset via TrackId
randy-seng Nov 17, 2023
fb68967
Move default track geometry factory method to pandas track storage
randy-seng Nov 20, 2023
f98da54
Implement contained by section method of TrackGeometryDataset
randy-seng Nov 20, 2023
000dbb2
Fix __len__ failing when PandasTrackDataset is empty
randy-seng Nov 21, 2023
68ce037
Don't store single detection tracks in TrackGeometryDataset
randy-seng Nov 21, 2023
d7226ba
Use more generic argument type
randy-seng Nov 21, 2023
9b77ec5
Add factory methods to return tracks as TrackDataset or list in GetAl…
randy-seng Nov 21, 2023
82de9eb
Use geometry operations of TrackDataset in TracksIntersectingSections…
randy-seng Nov 21, 2023
9a26e2b
Add function to separate sections by type
randy-seng Nov 22, 2023
91be3a4
Enable EventBuilder interface to add section id
randy-seng Nov 22, 2023
aa832a3
Return only tracks that have coordinates contained by section
randy-seng Nov 22, 2023
f61b356
Move responsibility to group sections by offset from TrackGeometryDat…
randy-seng Nov 27, 2023
6753719
Change intersection strategies to use TrackDataset geometry operations
randy-seng Nov 28, 2023
e0f5c52
Make track_ids property part of TrackGeometryDataset interface
randy-seng Nov 28, 2023
19578b0
Reuse existing geometry dataset when adding new tracks
randy-seng Nov 28, 2023
08c4e79
Move responsibility to manage track geometries for different offsets …
randy-seng Nov 28, 2023
52cf2c6
Set classification only once per track
randy-seng Nov 29, 2023
1f6eb37
Use coordinate with offset applied as event coordinate
randy-seng Nov 29, 2023
525ad2b
Fix unit test
randy-seng Nov 29, 2023
ef10f28
Fix unit test
randy-seng Nov 29, 2023
44cc708
Remove unused methods
randy-seng Nov 29, 2023
a45eae0
Remove IntersectionVisitor
randy-seng Nov 29, 2023
c06544e
Extend unit test for contained_by_sections
randy-seng Nov 29, 2023
215b7a0
Test offset applied when creating or adding to PygeosTrackGeometryDat…
randy-seng Nov 29, 2023
17c4ad4
Rename variable
randy-seng Nov 30, 2023
776261a
Collect fixtures to be used across several modules and move them to c…
randy-seng Nov 30, 2023
d36a4bd
Reuse existing TrackGeometryDataset when adding new tracks
randy-seng Nov 30, 2023
a16b945
Do not filter tracks with certain number of tracks when creating a ne…
randy-seng Nov 30, 2023
19e2956
Extend test cases for adding to a TrackDataset
randy-seng Nov 30, 2023
22a58e5
Refactor code duplication
randy-seng Nov 30, 2023
591338f
Reuse existing geometry datasets when removing tracks from TrackDataset
randy-seng Dec 1, 2023
84b6bc9
Fix broken dependencies in main application
randy-seng Dec 1, 2023
caf54b2
Add offset property to TrackGeometryDataset
randy-seng Dec 1, 2023
70e9525
Get track geometries only for a set of track ids
randy-seng Dec 1, 2023
6874841
Use factory method when creating PandasTrackDataset
randy-seng Dec 1, 2023
7f154eb
Reuse existing geometries when splitting TrackDatasets
randy-seng Dec 1, 2023
e7b343b
Use multi-index over TrackId and occurrence to speed up loading track…
randy-seng Dec 5, 2023
fbe5a7f
Disable filtering when trying to get all tracks
randy-seng Dec 5, 2023
4c7b97a
Fix getting all track ids from PandasTrackDataset
randy-seng Dec 5, 2023
748adca
Fix error caused by new DataFrame format used by PandasTrackDataset
randy-seng Dec 5, 2023
8c1cc25
Fix filtering DataFrame by occurrence
randy-seng Dec 5, 2023
2b8e45d
Move create intersection events use case from plugin to application l…
randy-seng Dec 6, 2023
bdc0cf0
Rename classes
randy-seng Dec 6, 2023
09af1a3
Let DataFrame filter predicate directly return filtered DataFrame ins…
randy-seng Dec 6, 2023
371c11d
Fix unit test
randy-seng Dec 6, 2023
5147b03
Use PandasTrackDataset as underlying implementation of track repository
randy-seng Dec 6, 2023
2718a02
Fix profile test
randy-seng Dec 6, 2023
646fe22
Merge branch 'main' into feature/2485-reimplement-intersection-strate…
randy-seng Dec 6, 2023
779302e
Switch to pandas track store implementation in benchmark
randy-seng Dec 7, 2023
0313b39
Use vectorized implementation of line_locate_point with dataframes
randy-seng Dec 11, 2023
3303795
Fix benchmarks
randy-seng Dec 11, 2023
33901fe
Merge branch 'main' into feature/2485-reimplement-intersection-strate…
randy-seng Dec 11, 2023
8ea26dc
Create geometries for offset before splitting TrackDatasets into batches
randy-seng Dec 11, 2023
84a938b
Merge remote-tracking branch 'origin/main' into feature/2485-reimplem…
briemla Dec 14, 2023
635b35f
Add vectorized projections calculation
briemla Dec 14, 2023
ae21332
Fix test to use newest track format
briemla Dec 18, 2023
a3c52c8
Parse multiple ottrk format versions
briemla Dec 18, 2023
4f3e2f3
Remove unused code
briemla Dec 18, 2023
b6c61ab
Ignore benchmark files
briemla Dec 18, 2023
921d570
Use dict comprehension
randy-seng Dec 19, 2023
37e2f75
Remove unused class
randy-seng Dec 19, 2023
b9f6e11
Remove callable variant to get tracks
randy-seng Dec 20, 2023
5eda2ec
Update documentation and change class name
randy-seng Dec 20, 2023
b6fc081
Raise error if dataframe don't have track id and occurrence set as mu…
randy-seng Dec 20, 2023
2225ec4
Fix typo
randy-seng Dec 20, 2023
a8c9b33
Change TrackDataFrameProvider implementation to use track id and occu…
randy-seng Dec 20, 2023
ccc5113
Merge branch 'main' into feature/2485-reimplement-intersection-strate…
randy-seng Dec 20, 2023
60426c4
Fix error message
briemla Dec 21, 2023
8d59daa
Move iteration over TrackDataset into SceneActionDetector
briemla Dec 21, 2023
85d416c
Prepare scene event creation for vectorized methods using a dataset
briemla Dec 21, 2023
62f5105
Move code to consume first and last track segments into TrackDataset
briemla Dec 21, 2023
c205f47
Rename parameters
briemla Dec 21, 2023
a856daa
Reduce number of generated flyweights to create scene events
briemla Dec 21, 2023
def5dd5
Change creation of events
briemla Dec 22, 2023
27b9814
Remove iterrows over dataframe
briemla Dec 22, 2023
32eaecc
Remove unnecessary mock
briemla Jan 9, 2024
6e2c13e
Move TrackRepository into separate module
briemla Jan 9, 2024
1461b13
Move TrackDataset into separate module
briemla Jan 9, 2024
8f56e69
Move TrackGeometryDataset into track_dataset module
briemla Jan 9, 2024
7c69210
Use correct type hint
briemla Jan 9, 2024
d4392b8
Remove unused code
briemla Jan 9, 2024
90eaaa5
Rename test file
randy-seng Jan 10, 2024
c3da16b
Remove resolved todo comment
randy-seng Jan 10, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
# OTAnalytics
data/
!tests/data
tests/data/OTCamera19_FR20_2023-05-24*
tests/scripts/

# Byte-compiled / optimized / DLL files
Expand Down
20 changes: 18 additions & 2 deletions OTAnalytics/application/analysis/intersect.py
Original file line number Diff line number Diff line change
@@ -1,9 +1,16 @@
from abc import ABC, abstractmethod
from typing import Iterable
from collections import defaultdict
from typing import Iterable, Mapping

from OTAnalytics.domain.event import Event
from OTAnalytics.domain.geometry import RelativeOffsetCoordinate
from OTAnalytics.domain.section import Section, SectionId
from OTAnalytics.domain.track import TrackId
from OTAnalytics.domain.types import EventType


class IntersectionError(Exception):
pass


class RunIntersect(ABC):
Expand All @@ -15,10 +22,19 @@ class RunIntersect(ABC):
@abstractmethod
def __call__(self, sections: Iterable[Section]) -> list[Event]:
raise NotImplementedError
# bla


class TracksIntersectingSections(ABC):
@abstractmethod
def __call__(self, sections: Iterable[Section]) -> dict[SectionId, set[TrackId]]:
raise NotImplementedError


def group_sections_by_offset(
sections: Iterable[Section], offset_type: EventType = EventType.SECTION_ENTER
) -> Mapping[RelativeOffsetCoordinate, list[Section]]:
grouped_sections: dict[RelativeOffsetCoordinate, list[Section]] = defaultdict(list)
for section in sections:
offset = section.get_offset(offset_type)
grouped_sections[offset].append(section)
return grouped_sections
3 changes: 2 additions & 1 deletion OTAnalytics/application/analysis/traffic_counting.py
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,8 @@
from OTAnalytics.domain.event import Event, EventRepository
from OTAnalytics.domain.flow import Flow, FlowRepository
from OTAnalytics.domain.section import Section, SectionId
from OTAnalytics.domain.track import TrackId, TrackRepository
from OTAnalytics.domain.track import TrackId
from OTAnalytics.domain.track_repository import TrackRepository
from OTAnalytics.domain.types import EventType

LEVEL_FROM_SECTION = "from section"
Expand Down
7 changes: 3 additions & 4 deletions OTAnalytics/application/datastore.py
Original file line number Diff line number Diff line change
Expand Up @@ -22,11 +22,10 @@
SectionListObserver,
SectionRepository,
)
from OTAnalytics.domain.track import (
TrackDataset,
from OTAnalytics.domain.track import TrackId, TrackImage
from OTAnalytics.domain.track_dataset import TrackDataset
from OTAnalytics.domain.track_repository import (
TrackFileRepository,
TrackId,
TrackImage,
TrackListObserver,
TrackRepository,
)
Expand Down
70 changes: 6 additions & 64 deletions OTAnalytics/application/eventlist.py
Original file line number Diff line number Diff line change
@@ -1,68 +1,11 @@
from typing import Iterable

from OTAnalytics.domain.event import Event, EventType, SceneEventBuilder
from OTAnalytics.domain.geometry import calculate_direction_vector
from OTAnalytics.domain.track import Track
from OTAnalytics.domain.event import Event
from OTAnalytics.domain.track_dataset import TrackDataset


class SceneActionDetector:
"""Detect when a road user enters or leaves the scene.

Args:
scene_event_builder (SceneEventBuilder): the builder to build scene events
"""

def __init__(self, scene_event_builder: SceneEventBuilder) -> None:
self._event_builder = scene_event_builder

def detect_enter_scene(self, track: Track) -> Event:
"""Detect the first time a road user enters the scene.

Args:
tracks (Track): the track associated with the road user

Returns:
Iterable[Event]: the enter scene event
"""
self._event_builder.add_event_type(EventType.ENTER_SCENE)
self._event_builder.add_road_user_type(track.classification)
first_detection = track.detections[0]
next_detection = track.detections[1]
self._event_builder.add_direction_vector(
calculate_direction_vector(
first_detection.x, first_detection.y, next_detection.x, next_detection.y
)
)
self._event_builder.add_event_coordinate(first_detection.x, first_detection.y)

return self._event_builder.create_event(first_detection)

def detect_leave_scene(self, track: Track) -> Event:
"""Detect the last time a road user is seen before leaving the scene.

Args:
tracks (Track): the track associated with the road user

Returns:
Iterable[Event]: the leave scene event
"""
self._event_builder.add_event_type(EventType.LEAVE_SCENE)
self._event_builder.add_road_user_type(track.classification)
last_detection = track.detections[-1]
second_to_last_detection = track.detections[-2]
self._event_builder.add_direction_vector(
calculate_direction_vector(
second_to_last_detection.x,
second_to_last_detection.y,
last_detection.x,
last_detection.y,
)
)
self._event_builder.add_event_coordinate(last_detection.x, last_detection.y)

return self._event_builder.create_event(last_detection)
"""Detect when a road user enters or leaves the scene."""

def detect(self, tracks: Iterable[Track]) -> list[Event]:
def detect(self, tracks: TrackDataset) -> list[Event]:
"""Detect all enter and leave scene events.

Args:
Expand All @@ -72,7 +15,6 @@ def detect(self, tracks: Iterable[Track]) -> list[Event]:
Iterable[Event]: the scene events
"""
events: list[Event] = []
for track in tracks:
events.append(self.detect_enter_scene(track))
events.append(self.detect_leave_scene(track))
tracks.apply_to_first_segments(events.append)
tracks.apply_to_last_segments(events.append)
return events
29 changes: 1 addition & 28 deletions OTAnalytics/application/geometry.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,36 +2,9 @@
from functools import singledispatchmethod
from typing import Generic, Iterable, TypeVar

from OTAnalytics.domain.geometry import (
Coordinate,
Line,
Polygon,
RelativeOffsetCoordinate,
)
from OTAnalytics.domain.section import Section
from OTAnalytics.domain.geometry import RelativeOffsetCoordinate
from OTAnalytics.domain.track import Track


class SectionGeometryBuilder:
def build_as_line(self, section: Section) -> Line:
return Line(section.get_coordinates())

def build_as_polygon(self, section: Section) -> Polygon:
return Polygon(section.get_coordinates())


class TrackGeometryBuilder:
def build(self, track: Track, offset: RelativeOffsetCoordinate) -> Line:
coordinates = [
Coordinate(
detection.x + offset.x * detection.w,
detection.y + offset.y * detection.h,
)
for detection in track.detections
]
return Line(coordinates)


LINE = TypeVar("LINE")
AREA = TypeVar("AREA")

Expand Down
6 changes: 2 additions & 4 deletions OTAnalytics/application/state.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,10 +17,8 @@
SectionRepositoryEvent,
SectionType,
)
from OTAnalytics.domain.track import (
Detection,
TrackId,
TrackImage,
from OTAnalytics.domain.track import Detection, TrackId, TrackImage
from OTAnalytics.domain.track_repository import (
TrackListObserver,
TrackObserver,
TrackRepository,
Expand Down
2 changes: 1 addition & 1 deletion OTAnalytics/application/use_cases/create_events.py
Original file line number Diff line number Diff line change
Expand Up @@ -110,7 +110,7 @@ def __init__(

def __call__(self) -> None:
"""Create scene enter and leave events and save them to the event repository."""
tracks = self._get_tracks()
tracks = self._get_tracks.as_dataset()
events = self._scene_action_detector.detect(tracks)
self._add_events(events)

Expand Down
Loading