Add support for local pmtiles tile package (via drag and drop) #1025

prusswan · 2025-01-23T17:59:45Z

This adds support for using local pmtiles tile packages that can be loaded via drag and drop (and possibly file picker). The behavior is very similar to https://pmtiles.io. These tile packages can already be referenced/used as a data source in the style, e.g. Tile URL: pmtiles://filename.pmtiles/{z}/{x}/{y}, just lacking a way to load the content.

Visual changes:

Added new spot in the top toolbar for dropping pmtiles package, also works with Inspect:

Launch Checklist

Briefly describe the changes in this PR.
Link to related issues.
Include before/after visuals or gifs if this PR includes visual changes.
Write tests for all new functionality.
Add an entry to CHANGELOG.md under the ## main section.

# Conflicts: # src/components/MapMaplibreGl.tsx

fixed TypeScript errors

codecov-commenter · 2025-01-24T04:23:51Z

Codecov Report

Attention: Patch coverage is 87.87879% with 4 lines in your changes missing coverage. Please review.

Project coverage is 64.46%. Comparing base (af01346) to head (cbbac9e).
Report is 59 commits behind head on main.

Files with missing lines	Patch %	Lines
src/components/App.tsx	33.33%	2 Missing ⚠️
src/components/MapMaplibreGl.tsx	88.88%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1025      +/-   ##
==========================================
+ Coverage   59.84%   64.46%   +4.61%     
==========================================
  Files         104      104              
  Lines        3011     5873    +2862     
  Branches      680     1728    +1048     
==========================================
+ Hits         1802     3786    +1984     
- Misses       1209     2085     +876     
- Partials        0        2       +2

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

src/components/AppToolbar.tsx

src/components/MapMaplibreGl.tsx

src/components/App.tsx

src/components/AppToolbar.tsx

src/components/MapMaplibreGl.tsx

src/components/AppToolbar.tsx

src/components/App.tsx

src/components/AppToolbar.tsx

src/components/App.tsx

HarelM · 2025-01-26T09:50:58Z

Looking at this holistically, I'm not sure this is the right way to achieve this.
Add sources as files may require support other file types like shp files, geojson etc, and the idea of this editor is to allow editing the style and I'm not sure it should handle loading of file like what is suggested here.
I don't know...

prusswan · 2025-01-26T10:30:30Z

Looking at this holistically, I'm not sure this is the right way to achieve this. Add sources as files may require support other file types like shp files, geojson etc, and the idea of this editor is to allow editing the style and I'm not sure it should handle loading of file like what is suggested here. I don't know...

I agree that this functionality is only feasible for PMTiles (up to a certain size) and is aided by the design and implementation of PMTiles library itself.

HarelM · 2025-01-26T20:01:50Z

I've posted a message on slack, let's see what the community thinks about this feature.

louwers

In my opinion there should be some way this feature is documented. Ideally it should be made discoverable. I suggest that when you drag anything onto the screen, you see a message saying what file types are supported. If you drag something that has an unsupported file type, you should get a message which file types are supported.

We can then extend the supported file types, for example, also allowing a style JSON to be dragged-and-dropped.

louwers

Should be tested with Cypress as well.

HarelM · 2025-01-27T12:14:15Z

I'm not sure It can be tested since dragging a file into the browser requires the ability to connect with the OS or something.
Not sure, but I won't be surprised if this can't be tested.

prusswan · 2025-01-27T13:50:37Z

In my opinion there should be some way this feature is documented. Ideally it should be made discoverable. I suggest that when you drag anything onto the screen, you see a message saying what file types are supported. If you drag something that has an unsupported file type, you should get a message which file types are supported.

I have set the file type to allow for .pmtiles only, this is reflected in the file picker. If invalid files are dragged instead, file rejection messages will be shown in console and alert dialog.

As for documentation, this can be a new section under the wiki?

louwers · 2025-01-27T23:31:21Z

@HarelM @prusswan https://github.com/cypress-io/cypress-example-recipes/tree/master/examples/testing-dom__drag-drop

HarelM · 2025-01-28T10:01:05Z

As far as I understand, and I might be wrong, this drag-and-drop is from inside the browser as opposed to this feature where you drag a file from the file viewer into the browser...

louwers · 2025-01-28T11:48:59Z

Sorry, should have been more specific. This one looks like it should work:

Selects a file or files in an HTML5 input element or simulates dragging a file or files into the browser.

HarelM · 2025-01-28T12:03:56Z

Ahh, found what you were referring to here in the Cypress docs:
https://docs.cypress.io/api/commands/selectfile

cy.document().selectFile('file.json', { action: 'drag-drop' })

prusswan · 2025-01-28T15:53:55Z

e2e test added: I wanted to test for popup but I simply could not get mouseover to work, so I settled for image diff. Hope this is not too big of change..

HarelM · 2025-01-28T19:12:17Z

cypress.config.ts

@@ -1,5 +1,6 @@
 import { defineConfig } from "cypress";
 import { createRequire } from "module";
+import getCompareSnapshotsPlugin from 'cypress-image-diff-js/plugin';


Is there a simpler solution without image diff? Checking that there's no error in case the file type is OK and check that the error appears when the file isn't?

src/locales/he/translation.json

HarelM · 2025-01-28T22:06:25Z

src/components/AppToolbar.tsx

+      return e.errors.map(f => f.message).join("\n")
+    }).join("\n");
+    console.error("Dropzone file rejected:", errorMessageLine);
+    alert(errorMessageLine);


I'm sure there are better ways to notify the user of a problem, not to mention this is not translated...

HarelM · 2025-01-28T22:07:19Z

src/components/AppToolbar.tsx

+    const errorMessageLine = r.map(e => {
+      return e.errors.map(f => f.message).join("\n")
+    }).join("\n");
+    console.error("Dropzone file rejected:", errorMessageLine);


I don't see a reason to both print to the console and notify the user, so this can be removed I guess.

HarelM · 2025-01-28T22:07:39Z

src/components/AppToolbar.tsx

+  }
+
+  onFileRejected = (r: FileRejection[]) => {
+    const errorMessageLine = r.map(e => {


Is it critical to collect all the error messages?

I can just show the first error and apply translation for it, if that is okay with you.

Are you planning on translating all the errors?
I would simply present a generic error message to the user about failure to open the provided file.

I think I can log the errors from react-dropzone to console, and show a translatable error message to the user.

HarelM · 2025-01-28T22:09:22Z

src/components/MapMaplibreGl.tsx

+      this.state.pmtilesProtocol!.add(file); // this is necessary for non-HTTP sources
+
+      if (map) {
+        file.getMetadata().then((metadata: any) => {


Does metadata has a better type? topojson maybe?

prusswan added 4 commits January 24, 2025 01:03

added react-dropzone

979137e

parse FileSource from dropped file for protocol.add

7ee8e44

# Conflicts: # src/components/MapMaplibreGl.tsx

fix for Inspect to work with FileSource

0fbda5d

removed unused prop (files in AppToolbar)

cdfe88f

fixed TypeScript errors

fixed lint errors

9db49ee

HarelM reviewed Jan 24, 2025

View reviewed changes

src/components/AppToolbar.tsx Outdated Show resolved Hide resolved

HarelM reviewed Jan 24, 2025

View reviewed changes

src/components/MapMaplibreGl.tsx Outdated Show resolved Hide resolved

HarelM reviewed Jan 24, 2025

View reviewed changes

src/components/App.tsx Outdated Show resolved Hide resolved

josxha reviewed Jan 24, 2025

View reviewed changes

src/components/AppToolbar.tsx Outdated Show resolved Hide resolved

src/components/MapMaplibreGl.tsx Outdated Show resolved Hide resolved

src/components/AppToolbar.tsx Outdated Show resolved Hide resolved

fixed i18n for PMTiles dropzone

486ede5

prusswan force-pushed the support-local-pmtiles branch from 28794e9 to b4e63e4 Compare January 24, 2025 08:18

prusswan added 2 commits January 24, 2025 16:24

cleaned up props name and console logging

7693217

make it clearer that the added protocol is for PMTiles only

a2809a9

prusswan force-pushed the support-local-pmtiles branch from b4e63e4 to a2809a9 Compare January 24, 2025 08:26

removed comment for react-dropzone import

d606781

HarelM reviewed Jan 26, 2025

View reviewed changes

src/components/AppToolbar.tsx Outdated Show resolved Hide resolved

HarelM reviewed Jan 26, 2025

View reviewed changes

src/components/App.tsx Outdated Show resolved Hide resolved

prusswan force-pushed the support-local-pmtiles branch from 921f304 to 29d78b2 Compare January 26, 2025 09:44

HarelM reviewed Jan 26, 2025

View reviewed changes

src/components/AppToolbar.tsx Outdated Show resolved Hide resolved

HarelM reviewed Jan 26, 2025

View reviewed changes

src/components/App.tsx Outdated Show resolved Hide resolved

HarelM approved these changes Jan 26, 2025

View reviewed changes

moved Dropzone's onDrop handler to AppToolbar

a2572c2

prusswan force-pushed the support-local-pmtiles branch from 29d78b2 to a2572c2 Compare January 26, 2025 09:52

louwers reviewed Jan 27, 2025

View reviewed changes

louwers requested changes Jan 27, 2025

View reviewed changes

limit accepted file types to .pmtiles only, improved error handling

d6cde2f

prusswan force-pushed the support-local-pmtiles branch from 9b0f37a to d6cde2f Compare January 27, 2025 13:40

prusswan requested a review from louwers January 28, 2025 15:55

HarelM reviewed Jan 28, 2025

View reviewed changes

prusswan force-pushed the support-local-pmtiles branch from 40bfbef to e32b006 Compare January 28, 2025 21:59

HarelM reviewed Jan 28, 2025

View reviewed changes

src/locales/he/translation.json Show resolved Hide resolved

HarelM reviewed Jan 28, 2025

View reviewed changes

src/locales/he/translation.json Show resolved Hide resolved

HarelM reviewed Jan 28, 2025

View reviewed changes

prusswan force-pushed the support-local-pmtiles branch from 087d113 to 27d8601 Compare January 28, 2025 23:05

added cypress e2e test, updated translation strings

cbbac9e

prusswan force-pushed the support-local-pmtiles branch from 27d8601 to cbbac9e Compare January 28, 2025 23:33

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add support for local pmtiles tile package (via drag and drop) #1025

Add support for local pmtiles tile package (via drag and drop) #1025

prusswan commented Jan 23, 2025

codecov-commenter commented Jan 24, 2025 •

edited

Loading

HarelM commented Jan 26, 2025

prusswan commented Jan 26, 2025

HarelM commented Jan 26, 2025

louwers left a comment •

edited

Loading

louwers left a comment

HarelM commented Jan 27, 2025

prusswan commented Jan 27, 2025

louwers commented Jan 27, 2025

HarelM commented Jan 28, 2025

louwers commented Jan 28, 2025

HarelM commented Jan 28, 2025

prusswan commented Jan 28, 2025 •

edited

Loading

HarelM Jan 28, 2025

HarelM Jan 28, 2025

HarelM Jan 28, 2025

HarelM Jan 28, 2025

prusswan Jan 28, 2025

HarelM Jan 28, 2025

prusswan Jan 28, 2025

HarelM Jan 28, 2025

Add support for local pmtiles tile package (via drag and drop) #1025

Are you sure you want to change the base?

Add support for local pmtiles tile package (via drag and drop) #1025

Conversation

prusswan commented Jan 23, 2025

Launch Checklist

codecov-commenter commented Jan 24, 2025 • edited Loading

Codecov Report

HarelM commented Jan 26, 2025

prusswan commented Jan 26, 2025

HarelM commented Jan 26, 2025

louwers left a comment • edited Loading

Choose a reason for hiding this comment

louwers left a comment

Choose a reason for hiding this comment

HarelM commented Jan 27, 2025

prusswan commented Jan 27, 2025

louwers commented Jan 27, 2025

HarelM commented Jan 28, 2025

louwers commented Jan 28, 2025

HarelM commented Jan 28, 2025

prusswan commented Jan 28, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codecov-commenter commented Jan 24, 2025 •

edited

Loading

louwers left a comment •

edited

Loading

prusswan commented Jan 28, 2025 •

edited

Loading