Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update - CKAN 2.10 compatibility and UI/Schemas/Profiles improvements #104

Merged
merged 83 commits into from
Sep 23, 2024

Commits on Aug 1, 2024

  1. Configuration menu
    Copy the full SHA
    49c4d24 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    2128d6c View commit details
    Browse the repository at this point in the history

Commits on Aug 4, 2024

  1. Add form tabs to dataset creation/editing datasets/resources

    This commit adds support for form tabs in the dataset creation and editing forms. The `package_form.html` template has been modified to include a "Next" button and the `save_button` block has been overridden to include the button. Additionally, the `config.py` file has been updated to add the necessary configuration options for form tabs. The `helpers.py` file has been modified to include helper functions related to form tabs, such as setting and retrieving the dataset form tabs.
    
    Fixes #87
    mjanez committed Aug 4, 2024
    Configuration menu
    Copy the full SHA
    2da1c1f View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    9bb1e8d View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    0e63426 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    f980580 View commit details
    Browse the repository at this point in the history

Commits on Aug 14, 2024

  1. Updated schema names and improved validators

    - Add subfield validators for legacy fields (new dcat schemas with dict objects for contact, publisher, etc.).
    - Rename fields according to new common schema names
    mjanez committed Aug 14, 2024
    Configuration menu
    Copy the full SHA
    34aedfb View commit details
    Browse the repository at this point in the history

Commits on Aug 22, 2024

  1. Add ckanext-dcat custom profiles

    - Add profiles for DCAT-AP, GeoDCAT-AP and NTI-RISP/DCAT (Spanish context).
    
    - Added new codelists generator/downloader to improve DCAT-AP mapping values.
    mjanez committed Aug 22, 2024
    Configuration menu
    Copy the full SHA
    d949571 View commit details
    Browse the repository at this point in the history
  2. Add SHACL tests for custom profiles

    -Add pytest-ckan, pytest-cov, and pytest-pretty to dev-requirements.txt
    -Remove flake8.yml workflow file
    -Add examples/ckan/dataset.json
    -Add examples/ckan/ckan_dataset.json
    -Add shacl import files for DCAT-AP 2.1.1 and 3.0.0
    mjanez committed Aug 22, 2024
    Configuration menu
    Copy the full SHA
    8f0e6f3 View commit details
    Browse the repository at this point in the history
  3. Fix F821 undefined name

    mjanez committed Aug 22, 2024
    Configuration menu
    Copy the full SHA
    bb9f4fd View commit details
    Browse the repository at this point in the history
  4. Update test.ini

    mjanez committed Aug 22, 2024
    Configuration menu
    Copy the full SHA
    b5dc788 View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    e1e488c View commit details
    Browse the repository at this point in the history
  6. Refactor dataset graph

    mjanez committed Aug 22, 2024
    Configuration menu
    Copy the full SHA
    f3f655f View commit details
    Browse the repository at this point in the history
  7. Configuration menu
    Copy the full SHA
    617d3ee View commit details
    Browse the repository at this point in the history
  8. Configuration menu
    Copy the full SHA
    d062580 View commit details
    Browse the repository at this point in the history
  9. Add reference and representation_type

    - Reference, relationship. https://semiceu.github.io/GeoDCAT-AP/releases/3.0.0/#Relationship
    - Spatial representation type” is meant mainly to describe the “method used to represent geographic information in the dataset”  https://semiceu.github.io/GeoDCAT-AP/releases/2.0.0/#spatial-representation-type-spatial-representation-type
    mjanez committed Aug 22, 2024
    Configuration menu
    Copy the full SHA
    2184aad View commit details
    Browse the repository at this point in the history
  10. Configuration menu
    Copy the full SHA
    5091c67 View commit details
    Browse the repository at this point in the history
  11. Fix test_shacl

    mjanez committed Aug 22, 2024
    Configuration menu
    Copy the full SHA
    3ff0417 View commit details
    Browse the repository at this point in the history
  12. Configuration menu
    Copy the full SHA
    af0710a View commit details
    Browse the repository at this point in the history
  13. Configuration menu
    Copy the full SHA
    a29d367 View commit details
    Browse the repository at this point in the history
  14. Configuration menu
    Copy the full SHA
    5d9a305 View commit details
    Browse the repository at this point in the history
  15. Configuration menu
    Copy the full SHA
    ba289eb View commit details
    Browse the repository at this point in the history
  16. Configuration menu
    Copy the full SHA
    ce772ba View commit details
    Browse the repository at this point in the history
  17. Todo legacy datasets

    mjanez committed Aug 22, 2024
    Configuration menu
    Copy the full SHA
    e635d71 View commit details
    Browse the repository at this point in the history
  18. Configuration menu
    Copy the full SHA
    2b4ddad View commit details
    Browse the repository at this point in the history
  19. Configuration menu
    Copy the full SHA
    13d12eb View commit details
    Browse the repository at this point in the history
  20. Configuration menu
    Copy the full SHA
    3eb49e7 View commit details
    Browse the repository at this point in the history
  21. Configuration menu
    Copy the full SHA
    29f80bd View commit details
    Browse the repository at this point in the history
  22. Configuration menu
    Copy the full SHA
    f1841e8 View commit details
    Browse the repository at this point in the history
  23. Fix rights bug

    mjanez committed Aug 22, 2024
    Configuration menu
    Copy the full SHA
    0b2d0cc View commit details
    Browse the repository at this point in the history

Commits on Aug 23, 2024

  1. Fix eu_dcat_ap profile

    mjanez committed Aug 23, 2024
    Configuration menu
    Copy the full SHA
    75a45bd View commit details
    Browse the repository at this point in the history
  2. Improve es_dcat (NTI-RISP) Profile

    A custom RDF profile based on the NTI-RISP for data portals in Spain
    Default values for some fields:
        ckanext-dcat/ckanext/dcat/.profiles.default_config.py
    
    More information and specification:
    https://datos.gob.es/es/documentacion/normativa-de-ambito-nacional
    https://datos.gob.es/es/documentacion/guia-de-aplicacion-de-la-norma-tecnica-de-interoperabilidad-de-reutilizacion-de
    mjanez committed Aug 23, 2024
    Configuration menu
    Copy the full SHA
    7a51f3d View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    dcd144e View commit details
    Browse the repository at this point in the history
  4. Refactor field mapping in SchemingDCATHarvester

    Simplify error reporting in the `apply_field_mapping` function by refactoring the code. Added a TODO comment to handle the case when `field_mapping` is not present.
    
    Fix schema validation in SchemingDCATCKANHarvester
    
    Ensure that the configured schema matches the local schema in the CKAN harvester. If they don't match, raise a `ValueError` with instructions to check the remote schema using the CKAN API or specify the correct local schema.
    
    Update 'allow_private_datasets' configuration in SchemingDCATCKANHarvester
    
    Check if 'allow_private_datasets' is present in the configuration and if it is a boolean. If not, set it to False. Also, raise a `ValueError` if 'api_key' is missing when using 'allow_private_datasets'.
    
    Set 'include_private' parameter in SchemingDCATCKANHarvester
    
    Set the 'include_private' parameter to False by default in the CKAN harvester. If 'allow_private_datasets' is set to True in the configuration, update the 'include_private' parameter to True.
    
    Update 'sort' and 'fq' parameters in SchemingDCATCKANHarvester
    
    Set the 'sort' parameter to "id asc" and update the 'fq' parameter based on the provided filter query terms in the CKAN harvester.
    mjanez committed Aug 23, 2024
    Configuration menu
    Copy the full SHA
    2386bc7 View commit details
    Browse the repository at this point in the history
  5. Add codelists for DCAT profiles

    - Improve cli to include download download_rdf_eu_vocabs
    mjanez committed Aug 23, 2024
    Configuration menu
    Copy the full SHA
    ff4de2c View commit details
    Browse the repository at this point in the history

Commits on Aug 27, 2024

  1. Configuration menu
    Copy the full SHA
    5c183ac View commit details
    Browse the repository at this point in the history
  2. Add docker-compose tests

    This repository includes a Docker Compose configuration to simplify running tests. The CKAN image is built using the Dockerfile located in the `docker/` directory.
    mjanez committed Aug 27, 2024
    Configuration menu
    Copy the full SHA
    9b16a8f View commit details
    Browse the repository at this point in the history
  3. Merge pull request #100 from mjanez/feature/docker-tests

    Add docker-compose tests
    mjanez authored Aug 27, 2024
    Configuration menu
    Copy the full SHA
    4b5c4b7 View commit details
    Browse the repository at this point in the history

Commits on Sep 10, 2024

  1. Fix search and index to ensure proper functionality

    - Fix before_dataset_index to use list instead of json lists stringify, remove empty extras keys and fix subfields.
    - Fix facet search and custom facets
    - Fix templates to include new helpers
    - Update search to be compatible with CKAN >2.10
    - Fix facet search operators.
    mjanez committed Sep 10, 2024
    Configuration menu
    Copy the full SHA
    c294495 View commit details
    Browse the repository at this point in the history
  2. Improve profiles and add DCAT-AP/GeoDCAT-AP 3

    - Align with ckanext-dcat refactorized profiles.
    - Add GeoDCAT-AP Profiles
    - Add DCAT-AP 3 profile
    - Improve ES DCAT-AP 2 profile.
    mjanez committed Sep 10, 2024
    Configuration menu
    Copy the full SHA
    7fb647a View commit details
    Browse the repository at this point in the history

Commits on Sep 16, 2024

  1. Improve schemas to align names with ckanext-dcat

    - Improve GeoDCAT-AP schemas (EU/ES) and update to GeoDCAT-AP 3.0.0 to match profiles.
    - Improve DCAT-AP schemas (EU/ES) and update to DCAT-AP 3.0.0 to match profiles.
    mjanez committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    0d6559b View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    8cb65ff View commit details
    Browse the repository at this point in the history
  3. Improve general UI, Home

    - Add new home to include detailed stats about portal and categories (themes) with datasets.
    - Improve header/footer. Move language selector and fix responsive of general web.
    mjanez committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    4810b07 View commit details
    Browse the repository at this point in the history
  4. Update package/resource forms to Tabs

    Metadata creation/editing forms now use tabs instead of steps, allowing all metadata elements to be set as mandatory and greatly improving the interaction and usability of the interface.
    mjanez committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    ef76bb3 View commit details
    Browse the repository at this point in the history
  5. Improve i18n and icons

    mjanez committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    4dca2e4 View commit details
    Browse the repository at this point in the history
  6. Fix and improve templates for CKAN 2.10

    - Forms that are submitted via JavaScript modules also need to submit the CSRF token. `{{ h.csrf_input() if 'csrf_input' in h }}` to compat with <2.10 versions.
    - Bootstrap 5 data attributes: `data-module` to `data-bs`
    - Fix group_items type template.
    - Improve endpoints index cards (/endpoints).
    - Used jinja truncate filter instead of helper.
    mjanez committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    c7aabef View commit details
    Browse the repository at this point in the history
  7. Configuration menu
    Copy the full SHA
    f358203 View commit details
    Browse the repository at this point in the history
  8. Fix before_update

    mjanez committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    7c303d9 View commit details
    Browse the repository at this point in the history
  9. Improve package_item.html template

    - Update badge classes for draft, metadata template, and deleted states
    - Update class for private dataset badge
    - Update link truncation for dataset title
    - Update data attribute for resource format labels
    mjanez committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    fdf55e8 View commit details
    Browse the repository at this point in the history
  10. Improve base admin config template

    - Remove unnecesary Homepage selector.
    - Fix POST method errors when upload local logos.
    mjanez committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    a344aec View commit details
    Browse the repository at this point in the history
  11. Update configuration and add stats init to plugin

    - Add cache config vars.
    mjanez committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    006078e View commit details
    Browse the repository at this point in the history
  12. Configuration menu
    Copy the full SHA
    3da105a View commit details
    Browse the repository at this point in the history
  13. Refactor download metadata template and improve metadata profiles

    The download_metadata.html template has been refactored to remove unnecessary code and improve the organization of metadata profiles. The changes include:
    - Simplifying the inspire_metadata_values set by using the h.schemingdcat_get_inspire_dcat_types() helper function
    - Updating the logic to generate the profiles list by including additional endpoint properties like profile_info_url and profile_label_order
    - Sorting the profiles list based on the profile_label_order attribute
    - Modifying the rendering of download links in the dropdown menu by creating a render_download_link macro
    - Adding logic to display CSW geospatial metadata links only when the package's dcat_type is in the inspire_metadata_values set
    - Adding logic to display RDF DCAT linked data links based on the package's theme and the profile's profile_label
    
    These changes improve the code structure and enhance the usability of the download metadata functionality.
    mjanez committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    6954453 View commit details
    Browse the repository at this point in the history
  14. Improve metadata_info and profile badges

    - Add truncate-text class to avoid problems when using in mobile or metadata elements with large texts.
    - Improve badges for profiles like DCAT, DCAT-AP, GeoDCAT-AP, NTI-RISP, ...
    mjanez committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    0ffa59e View commit details
    Browse the repository at this point in the history
  15. Configuration menu
    Copy the full SHA
    c4d131b View commit details
    Browse the repository at this point in the history
  16. Configuration menu
    Copy the full SHA
    e136909 View commit details
    Browse the repository at this point in the history
  17. Add xls_template_config

    First approach
    mjanez committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    69e04f3 View commit details
    Browse the repository at this point in the history
  18. Improve style to improve header language selector dropdown menu

    - Prevents languages from being wrapped on multiple lines.
    mjanez committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    66dc66d View commit details
    Browse the repository at this point in the history
  19. Configuration menu
    Copy the full SHA
    1af58e1 View commit details
    Browse the repository at this point in the history

Commits on Sep 18, 2024

  1. Fix Solr indexing by converting dict fields to JSON strings

    - Added `_before_index_dump_dicts` method to `PackageController` class to convert dict fields in the data dictionary to JSON strings.
    - Updated `before_dataset_index` method to call `_before_index_dump_dicts` to ensure all fields can be indexed by Solr.
    - Enhanced docstring for `_before_index_dump_dicts` to explain the necessity of converting dict fields to JSON strings, referencing related issues and errors.
    - Ensured that all fields in the data dictionary are in a format that Solr can handle, preventing errors such as "missing required field" even when the field is present.
    - Addressed issues observed in CKAN versions 2.10.4 and Solr 9, where attempts to upload resources to the Datastore resulted in errors due to the presence of dict fields in the data dictionary (ckan/ckan#8423).
    - Referenced related issues: CKAN - Custom plugin/theme error datastore using fluent presets #7750 and Solr error: missing required field #7730.
    mjanez committed Sep 18, 2024
    Configuration menu
    Copy the full SHA
    fb8abf6 View commit details
    Browse the repository at this point in the history
  2. Update open data stats: Skip statistics update during DB initializati…

    …on or upgrade
    
    - Added a check to determine if a database initialization or upgrade command is being executed.
    - Skipped the statistics update during these operations to avoid potential conflicts.
    - Ensured that the statistics update only runs when it is safe to do so, preventing issues during database setup or migration.
    mjanez committed Sep 18, 2024
    Configuration menu
    Copy the full SHA
    e3bb1ee View commit details
    Browse the repository at this point in the history
  3. Improve form tabs

    - Fix Next button function.
    mjanez committed Sep 18, 2024
    Configuration menu
    Copy the full SHA
    0cbb625 View commit details
    Browse the repository at this point in the history
  4. Fix package snippets

    mjanez committed Sep 18, 2024
    Configuration menu
    Copy the full SHA
    448113e View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    4b10208 View commit details
    Browse the repository at this point in the history
  6. Fix spatial_query snippet bugs

    Unnecesary update to data-bs boostrap class.
    mjanez committed Sep 18, 2024
    Configuration menu
    Copy the full SHA
    3026467 View commit details
    Browse the repository at this point in the history
  7. Configuration menu
    Copy the full SHA
    2c02458 View commit details
    Browse the repository at this point in the history
  8. Fix tags containers

    mjanez committed Sep 18, 2024
    Configuration menu
    Copy the full SHA
    50945d0 View commit details
    Browse the repository at this point in the history
  9. Configuration menu
    Copy the full SHA
    537504c View commit details
    Browse the repository at this point in the history

Commits on Sep 19, 2024

  1. Update version to 4.0.0

    mjanez committed Sep 19, 2024
    Configuration menu
    Copy the full SHA
    767fa65 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    bac0862 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    6dc1e4c View commit details
    Browse the repository at this point in the history
  4. Improve tests

    mjanez committed Sep 19, 2024
    Configuration menu
    Copy the full SHA
    f2a0ed8 View commit details
    Browse the repository at this point in the history

Commits on Sep 20, 2024

  1. Removed unnecessary data-bs attributes for custom slug-preview module

    The data-bs attributes were removed from the templates as they are specific to Bootstrap and not required for the custom slug-preview module.
    mjanez committed Sep 20, 2024
    Configuration menu
    Copy the full SHA
    ddc4de6 View commit details
    Browse the repository at this point in the history
  2. Fix api button accordions

    - Add R and curl from older extension.
    mjanez committed Sep 20, 2024
    Configuration menu
    Copy the full SHA
    fcec54b View commit details
    Browse the repository at this point in the history
  3. Add highlighting and quote alerts

    - Avalaible values are 'tip', 'note', 'warning', 'caution', 'important'
    
      Example:
          {% block api_info %}
            {% set quote_content %}
              {% trans %}
              Further information in the <a href="http://docs.ckan.org/en/latest/maintaining/datastore.html" target="_blank">main CKAN Data API and DataStore documentation</a>.
              {% endtrans %}
            {% endset %}
            {% snippet "schemingdcat/snippets/quote_alerts.html", alert_type='tip', content=quote_content %}
          {% endblock %}
    mjanez committed Sep 20, 2024
    Configuration menu
    Copy the full SHA
    eefdcac View commit details
    Browse the repository at this point in the history
  4. Improve admin pages

    mjanez committed Sep 20, 2024
    Configuration menu
    Copy the full SHA
    53f9dc1 View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    8f22343 View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    1e504ca View commit details
    Browse the repository at this point in the history
  7. Configuration menu
    Copy the full SHA
    32300b8 View commit details
    Browse the repository at this point in the history

Commits on Sep 21, 2024

  1. Configuration menu
    Copy the full SHA
    4bfcea4 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    e3d81ff View commit details
    Browse the repository at this point in the history

Commits on Sep 22, 2024

  1. Improve README images

    mjanez committed Sep 22, 2024
    Configuration menu
    Copy the full SHA
    5cd2bef View commit details
    Browse the repository at this point in the history

Commits on Sep 23, 2024

  1. Merge pull request #101 from mjanez/feature/2.10-update

    Update - CKAN 2.10 compatibility and UI/Schemas/Profiles improvements
    mjanez authored Sep 23, 2024
    Configuration menu
    Copy the full SHA
    3a68063 View commit details
    Browse the repository at this point in the history