Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Time serie selection handled by the advanced metadata panel #9821

Closed
mattiagiupponi opened this issue Aug 5, 2022 · 7 comments
Closed

Time serie selection handled by the advanced metadata panel #9821

mattiagiupponi opened this issue Aug 5, 2022 · 7 comments

Comments

@mattiagiupponi
Copy link
Contributor

mattiagiupponi commented Aug 5, 2022

The time selection is going to be added to the advanced metadata page in the settings panel. If the has_time is selected a new panel for the time series settings is open:
image

Based on the selection of the presentation, two new input boxes are loaded to let the user configure the time series:
image
The ? open a modal with some information about "how to configure" the time series
image
Internally the dataset_metadata view will communicate with GeoServer to configure the time series. If the has_time is removed, also the configuration in GeoServer is disabled.
image

@mattiagiupponi mattiagiupponi changed the title Move the TimeSerie selection from FE to the advanced metadata panel Time serie selection handled by the advanced metadata panel Aug 5, 2022
@mattiagiupponi
Copy link
Contributor Author

mattiagiupponi commented Aug 18, 2022

After some deeper analysis and work on this feature, some issues occurred.
Let's suppose that we want to upload a new dataset with a time series (in which only the year is available) geoserver save the feature type as Integer:
image

As an instance, the time series cannot be enabled since geoserver is not able to find any data type:

image

The actual workflow with the initial step forces the type conversion during the import to let the type be stored as expected.

@giohappy @etj any thoughts?

@giohappy
Copy link
Contributor

@mattiagiupponi that's too bad. I suppose GeoNode importer configures the dateformattransform transformation.

The new importer shouldn't have this problem, since we publish from the DB and the field type should be automatically recognized. Do you confirm @mattiagiupponi?

For the gs-importer case we only have two options I guess:

  • guess datetime fields (as the import procedure does now) and automatically configure the transformation
  • make the client guess datetime fields and let the user select them, once the client-side upload configuration tool will be in place

The automatic parsing of datetime fields on the BE could leverage the current dataset_eligible_for_time_dimension. It's not very robust, since it only read the first feature to guess the field format, but it could be probably improved a bit to make it more robust (if the first feature's value is empty try another one, maybe samples from the first 10 features).

Since we're going to adopt the new importer for all the vector formats soon, I would consider solution 1 (if it doesn't take too much)

@mattiagiupponi
Copy link
Contributor Author

mattiagiupponi commented Aug 24, 2022

The new importer shouldn't have this problem, since we publish from the DB and the field type should be automatically recognized. Do you confirm @mattiagiupponi?

The new importer will take the field type based on how ogr2ogr reads the dataset's schema. But I guess that we will continue to save the data as "Integer" and not as "date". I can try it on local to have an answer on this.

  • make the client guess datetime fields and let the user select them, once the client-side upload configuration tool will be in place
    Since we're going to adopt the new importer for all the vector formats soon, I would consider solution 1 (if it doesn't take too much)

I think this is not strictly needed.
In my opinion, the dataset should be imported as is, as we already discuss the time series should be set later via the settings.
In the "Attribute" input box in the new settings panel, the user should be able to choose also the int value, then ones are selected, the BE should

  • change (if possible) the feature type from INT to Datetime. If not raise an error to the user
  • call GeoServer to reload the feature type schema and empty the cache
  • set the time-series
  • return the status to the user

I would avoid too much guessing from the client side.
Is this may be related to this one geosolutions-it/geoserver#339 @giohappy?

@afabiani
Copy link
Member

@mattiagiupponi the selectbox should recognize Integer fields too IMHO. Consider the case when we have just the year as an instance. This should be also the current GeoServer behavior.

@mattiagiupponi
Copy link
Contributor Author

@mattiagiupponi the selectbox should recognize Integer fields too IMHO. Consider the case when we have just the year as an instance. This should be also the current GeoServer behavior.

Indeed @afabiani , this phrase is no longer valid:

In the attribute and end_attribute fields, only the xsd:dateTime fields are displayed.

@mattiagiupponi mattiagiupponi self-assigned this Aug 25, 2022
@giohappy
Copy link
Contributor

The final proposal is the following.

CURRENT IMPORTER:
for the moment we keep the time step during the upload.

NEW IMPORTER:
Initially, only file format with Date / DateTIme field type will be supported (ESRI Shapefile and GPKG).
For the rest (textual formats like CSV and GeoJSON) the following will be estimated:

  1. Create a new functionality in GeoNode to create derived/computed fields. This feature can be used to create new Time fields from existing fields. This means implementing the following:
  • add a new column to the data table
  • update the dynamic model schema accordingly
  • update the Geoserver FeatureType schema
  1. Expose this functionality inside the Metadata attributes list as well as a new dedicated API endpoint.

Once the new time field will be available in Geoserver, the field will appear inside the TIme dimension management box under the GeoNode's dataset metadata settings (already implemented).

mattiagiupponi added a commit that referenced this issue Aug 25, 2022
github-actions bot pushed a commit that referenced this issue Aug 26, 2022
…anel (#9840)

* [Fixes #9821] Time serie selection handled by the advanced metadata panel

* [Fixes #9821] add new attribute to time series

* [Fixes #9821] add new attribute to time series

* [Fixes #9821] add new attribute to time series

* [Fixes #9821] add new attribute to time series

* [Fixes #9821] add new attribute to time series

* [Fixes #9821] add new attribute to time series

* [Fixes #9821] add new attribute to time series

* [Fixes #9821] add new attribute to time series

* [Fixes #9821] add new attribute to time series

* [Fixes #9821] Time serie selection handled by the advanced metadata panel

* [Fixes #9861]Cannot open metadata detail with thesaurus installed

* [Fixes #9821] Time serie selection handled by the advanced metadata panel

* [Fixes #9821] Time serie selection handled by the advanced metadata panel

* [Fixes #9821] Fix wrong subtype in case of timeserie dataset

* [Fixes #9821] Fix circleci build
@afabiani afabiani added the 4.0.x label Aug 26, 2022
@giohappy giohappy reopened this Aug 31, 2022
@giohappy
Copy link
Contributor

@marthamareal please test this on the development server.

1 - Test that the configuration is correctly shown when you upload a new layer and configure it as time series
2 - Test that you can configure a time series afterward. You will have to upload a layer without configuring it as a time series dataset. Keep in mind that for testing this you need to upload a Shapefile having a Date field in it, otherwise the configuration panel won't find any suitable field to activate the time dimension. If you try with simple String and Number fields no field will be recognized as suitable.

These tests must be added to Cucumber.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants