-
Notifications
You must be signed in to change notification settings - Fork 1.3k
[core, ios, macos, android] Support TinySDF for local glyph generation #10522
Conversation
LGTM! |
All right, I think this is ready for iOS and Android work to start. Since core behavior won't actually change without a platform-specific implementation, I might hold off trying to merge it until the first of the platform implementations is close to ready. @boundsj, you mentioned that you might consider trying to do this work on the On the core side, the only testing is the @mourner Can you review the tiny_sdf implementation? /cc @ivovandongen b/c I remember you were interested in the Android side of this when I first started looking at doing it back in June. /cc @nickidlugash and @lilykaiser for 🇨🇳 product design expertise |
b1d9da7
to
6caccf6
Compare
Really excited to see progress toward #7862, even if it ends up being limited to CJK fonts for the time being.
On Apple platforms, font selection is a bit more nuanced than searching for keywords inside PostScript font identifiers. In part, this is because non-Western fonts (especially East Asian fonts) don’t necessarily have the same variants as Western fonts. In C, the entry point for font selection is Core Text’s
Yes, on Apple platforms, it’s straightforward to get the glyph metrics using Outside of CJK, the main benefit would be the ability to use local or system fonts (#7862). That would result in large savings to offline download sizes. |
@ChrisLoer the TinySDF part of the PR looks great to me. |
6caccf6
to
6ac47a9
Compare
Interesting. That's a lot of granularity, although I can see how it would be useful. I worry about how much we end up forcing people to implement their style in multiple places (i.e. one part in the style itself via Studio, the other part in their app using the SDK). Are there other situations where we have this kind of split, with platform-specific style options being pushed down to SDK settings? Would it ever make sense to push platform-specific style options into the style spec (so instead of just To keep the scope of an initial implementation limited to the CJK case (and provide an interface similar to what we have for GL JS), it sounds to me like it would work to introduce a |
6ac47a9
to
e4e6aaa
Compare
I suggested NSFontDescriptor/UIFontDescriptor based on seeing the parameters offered by TinySDF. However, it looks like GL JS only exposes the
If we expose a per-layer option (as opposed to a global option), then I don’t see a problem with forcing cross-platform developers to implement things slightly differently on each platform. After all, the runtime styling API already differs across platforms in terms of syntax and terminology, and local fonts behave differently on every platform anyways.
In principle, the style JSON format should be platform-agnostic. (For this reason, I’d like for us to discourage developers from using the |
fd41cdc
to
ac8c890
Compare
I merged in the macOS/iOS implementation from There are loose ends that need addressing (and I'll need help from someone experienced with iOS):
I'll start exploring an Android implementation now. |
I've started work on an Android implementation based off this PR -- it lives at PR #10608. |
Adding a MGLIdeographicFontFamilyName to the containing app's Info.plist will result in CJK glyphs being rasterized on demand (#10522)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This guide to common Core Text font operations may contain some leads.
platform/macos/src/MGLMapView.mm
Outdated
@@ -252,7 +252,7 @@ + (NSArray *)restorableStateKeyPaths { | |||
return @[@"camera", @"debugMask"]; | |||
} | |||
|
|||
- (void)commonInit { | |||
- (void)commonInit:(nullable NSString*)fontFamily { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It needs to be possible to set the font family at an arbitrary time after creating the mbgl::Renderer
. The reason this method doesn’t take any parameters – not even the view’s frame – is that we have no control over the parameters at initialization when the view is initialized from a XIB or storyboard.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
#10612 would address the immediate issue here pretty elegantly, although there would remain room for future improvement.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
👍
platform/macos/src/MGLMapView.mm
Outdated
@@ -274,7 +274,7 @@ - (void)commonInit { | |||
|
|||
_mbglThreadPool = mbgl::sharedThreadPool(); | |||
|
|||
auto renderer = std::make_unique<mbgl::Renderer>(*_mbglView, [NSScreen mainScreen].backingScaleFactor, *mbglFileSource, *_mbglThreadPool, mbgl::GLContextMode::Unique); | |||
auto renderer = std::make_unique<mbgl::Renderer>(*_mbglView, [NSScreen mainScreen].backingScaleFactor, *mbglFileSource, *_mbglThreadPool, mbgl::GLContextMode::Unique, mbgl::optional<std::string>(), fontFamily ? std::string([fontFamily UTF8String]) : mbgl::optional<std::string>()); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Any changes to this file should also be reflected in MGLMapView.mm in the iOS project.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@akitchen has my back. 😉
platform/macos/src/MGLMapView.mm
Outdated
@@ -252,7 +252,7 @@ + (NSArray *)restorableStateKeyPaths { | |||
return @[@"camera", @"debugMask"]; | |||
} | |||
|
|||
- (void)commonInit { | |||
- (void)commonInit:(nullable NSString*)fontFamily { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Per #10522, I think NSFontDescriptor/UIFontDescriptor is the more forward-compatible datatype for this option. It’s important to consider forward compatibility for iOS especially, because changing the type of a property or parameter will require a semver-major release.
If we aren’t ready to support the many options in NSFontDescriptor/UIFontDescriptor, we can specify in the documentation that only the font family option is supported for now.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, it’s good enough for now. In the future, if we open up more options, then we can introduce formal APIs for them without worrying about backwards compatibility.
platform/macos/src/MGLMapView.mm
Outdated
@@ -252,7 +252,7 @@ + (NSArray *)restorableStateKeyPaths { | |||
return @[@"camera", @"debugMask"]; | |||
} | |||
|
|||
- (void)commonInit { | |||
- (void)commonInit:(nullable NSString*)fontFamily { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In terms of plumbing the value through mbgl, consider adding an overridable method to mbgl::RendererFrontend
that returns a CTFontDescriptor. The Darwin implementation of MGLRenderFrontend can convert to and from NSFontDescriptor or UIFontDescriptor, using typedefs and conditional compilation to paper over differences between iOS and macOS.
Since CTFontDescriptor is itself platform-specific, consider wrapping it in a new mbgl::FontDescriptor
class along the lines of mbgl::RendererFrontend
before returning it to mbgl::RendererFrontend
.
constexpr float light = -.4; | ||
constexpr float regular = 0.0; | ||
constexpr float medium = .23; | ||
constexpr float bold = .4; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Would it be possible to include CFFontTraits.h for CTFontSymbolicTraits
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In terms of weight, the only thing CTFontSymbolTraits
exposes is kCTFontTraitBold
. Or did you have something else in mind? I did try using that to load a bold font without luck...
double totalAdvance = CTFontGetAdvancesForGlyphs(font, kCTFontOrientationDefault, glyphs, advances, 1); | ||
|
||
// Break in the debugger to see these values: translating from "user coordinates" to bitmap pixel coordinates | ||
// should be OK, but a lot of glyphs seem to have empty bounding boxes...? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Are you seeing this even if you make the boundingRects
and advances
arrays the same length as glyphCount
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The assert(glyphCount == 1);
never fires, so yes, the arrays should be the same length.
Adding a MGLIdeographicFontFamilyName to the containing app's Info.plist will result in CJK glyphs being rasterized on demand (#10522)
a7e5181
to
a648a9b
Compare
Adding a MGLIdeographicFontFamilyName to the containing app's Info.plist will result in CJK glyphs being rasterized on demand (#10522)
…fic than the title.
…code. Add local font family to default iosapp configuration.
Also removes related dead code in macos MGLMapView.mm
828243e
to
e91272d
Compare
OK, I think this branch may be ready to merge. I've modestly squashed the commit history to get rid of obvious fixups, but I left the iOS attempts at getting glyph metrics and setting font weights in the history, I also didn't squash into @akitchen 's or @asheemmamoowala 's commits so that they get blamed appropriately. 😉 Here's the current state of all the compromises involved:
Both platforms have API-level documentation. After this PR merges, we should consider platform-specific "Examples" analogs to the GL JS example. @lilykaiser can you help keep track of this loose end? I'm happy to work on the examples but I don't know where they live for the various platforms, when/how we generate them, etc. This PR would be a good candidate for some device-specific testing as part of the release process -- I've relied on using emulators for testing the iOS and Android code. @akitchen / @zugaldia is there some way we can put "exercise this functionality on our local devices" on part of a todo-list for testing the versions of the SDKs that incorporate this functionality? @ivovandongen and @1ec5, I'll wait for 🍏 from both of you before merging -- thanks for all your help in getting this ready, I know both of you have been busy working against other deadlines. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
iOS and macOS changes look fine, other than the naming issue. Thanks @akitchen for sorting that out.
Would you do the honors and add a blurb in the iOS and macOS changelogs?
Map snapshots won’t benefit from this feature yet, correct?
@@ -19,3 +19,7 @@ The default value is `https://api.mapbox.com`. | |||
## MGLMapboxMetricsEnabledSettingShownInApp | |||
|
|||
If you have implemented custom opt-out of Mapbox Telemetry within the user interface of your app, use this key to disable the built-in check for opt-out support. See [this guide](https://www.mapbox.com/ios-sdk/#telemetry_opt_out) for more details. | |||
|
|||
## MGLIdeographFontFamilyName |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
“Ideograph font” is ungrammatical and sounds funny. The previous terminology, “ideographic font” was more correct. If inconsistency between the iOS/macOS map SDK’s public API and the underlying types is a problem – it rarely is – can we rename the latter?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reverted to "ideographic"
|
||
## MGLIdeographFontFamilyName | ||
|
||
The name of the font family to use for client-side text rendering of CJK ideographs. Set this to the name of a font family which will be available at run time, e.g. `PingFang`. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
iOS and macOS both come with PingFangHK
, PingFangSC
, and PingFangTC
. I wonder which font PingFang
ends up loading.
Original GL JS name was meant to represent "font family to use for locally generating ideographs", but "ideographic font family" communicates a similar intent more concisely.
I'm not sure which iOS and Android releases this will be part of, but here's a blurb that we can add to the changelog once we know:
Good point regarding map snapshots -- they're not hooked up now, but it should be easy to do. I split that out into #10665. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good.
import com.mapbox.mapboxsdk.maps.OnMapReadyCallback; | ||
import com.mapbox.mapboxsdk.testapp.R; | ||
|
||
public class LocalGlyphActivity extends AppCompatActivity { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could use a short JavaDoc explanation what we're testing
@@ -0,0 +1,31 @@ | |||
#pragma once |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is now inconsistently named with the implementation file.
Adding a MGLIdeographicFontFamilyName to the containing app's Info.plist will result in CJK glyphs being rasterized on demand (#10522)
Adding a MGLIdeographicFontFamilyName to the containing app's Info.plist will result in CJK glyphs being rasterized on demand (#10522)
Adding a MGLIdeographicFontFamilyName to the containing app's Info.plist will result in CJK glyphs being rasterized on demand (#10522)
This is a core/darwin/android port of mapbox/mapbox-gl-js#4895, which uses https://github.com/mapbox/tiny-sdf in combination with locally available fonts to generate SDF for ideographic characters. This makes a dramatic improvement in loading times for maps that contain lots of CJK characters.
Qt is left out for now, but this PR establishes the pattern to follow if we find demand for a Qt implementation.
The branch currently contains:
GlyphManager
to allow it to load local versions of glyphs (and transform them into SDFs) if they're availableLocalGlyphRasterizer
that needs platform specific font loading and rasterization implementationsLocalGlyphRasterizer
that just returns a pre-rasterized '中' for any glyph that allows ideographic breaking.
- A darwin implementation that renders CJK glyphs using CoreText - An android implementation that renders CJK glyphs using `android.graphics.Canvas`I've lightly tested these builds locally with macosapp, mbgl-glfw, the ios app running on my iPhone, and the Android test app running on an emulated Pixel. I've only tested installing fonts on OS X -- on the phones I just rely on whatever happens to be installed.
TODO:
Darwin font weight support (current implementation doesn't seem to work)Out of scope:
/cc @boundsj @kkaefer @chriswu42 @lilykaiser @ivovandongen @1ec5 @tobrun @akitchen @zugaldia