-
Notifications
You must be signed in to change notification settings - Fork 11
/
Copy pathOverview.bs
3004 lines (2392 loc) · 141 KB
/
Overview.bs
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
<pre class='metadata'>
Title: Incremental Font Transfer
Shortname: IFT
Status: ED
Prepare for TR: no
Group: webfontswg
Level: none
Markup Shorthands: css no
TR: https://www.w3.org/TR/IFT/
ED: https://w3c.github.io/IFT/Overview.html
Editor: Chris Lilley, W3C, https://svgees.us/, w3cid 1438
Editor: Garret Rieger, Google Inc., grieger@google.com, w3cid 73905
Editor: Skef Iterum, Adobe Inc., siterum@adobe.com, w3cid 137857
Abstract: This specification defines a method to incrementally transfer a font
from server to client. The client loads only the portion(s) of the font that
they actually need, significantly reducing data transfer. Multiple incremental
additions to the same font are possible, such as a user agent updating a font
as a user browses multiple pages. Incremental transfer improves on
unicode-range by avoiding damage to layout (kerning, ligatures, etc) rules,
meaning it can efficiently support fine grained increments to latin and
to complex scripts like Indic or Arabic.
</pre>
<!--
for things that are not in specref
https://www.specref.org/
-->
<pre class=link-defaults>
spec:fetch; type:dfn; for:/; text:status
spec:fetch; type:dfn; for:/; text:response
</pre>
<pre class=biblio>
{
"PFE-report": {
"href": "https://www.w3.org/TR/PFE-evaluation/",
"authors": ["Chris Lilley"],
"status": "Note",
"publisher": "W3C",
"title": "Progressive Font Enrichment: Evaluation Report",
"date": "15 October 2020"
},
"Shared-Brotli": {
"href": "https://datatracker.ietf.org/doc/html/draft-vandevenne-shared-brotli-format-09",
"authors": [
"J. Alakuijala",
"T. Duong",
"R. Obryk",
"Z. Szabadka",
"L. Vandevenne"
],
"status": "Internet Draft",
"title": "Shared Brotli Compressed Data Format",
"date": "Sep 2022"
},
"open-type": {
"href": "https://docs.microsoft.com/en-us/typography/opentype/spec",
"authors": [],
"status": "Note",
"publisher": "Microsoft",
"title": "OpenType Specification",
"date": "May 2024"
},
"fetch": {
"href": "https://fetch.spec.whatwg.org/",
"authors": [],
"status": "Living Standard",
"publisher": "What WG",
"title": "Fetch Standard",
"date": "17 June 2024"
},
"enabling-typography": {
"href": "https://tiro.com/John/Enabling_Typography_(OTL).pdf",
"authors": ["John Hudson"],
"status": "Note",
"publisher": "John Hudson",
"title": "Enabling Typography: towards a general model of OpenType Layout",
"date": "April 15th, 2014"
}
}
</pre>
<style>
.conform:hover {background: #31668f; color: white}
.conform:target {padding: 2px; border: 2px solid #AAA; background: #31668f; color: white }
table {
width: 100%;
}
table, tr {
border: 1px solid #aaa;
border-collapse: collapse;
}
th, td {
padding: 0.5rem;
}
</style>
Introduction {#intro}
=====================
<em>This section is not normative.</em>
Incremental Font Transfer (IFT) is a technology to improve the latency of remote fonts (or "web fonts") on
the web. Without this technology, a browser needs to download every last byte of a font before it can render
any characters using that font. IFT allows the browser to download only some of the bytes in the file, thereby
decreasing the perceived latency between the time when a browser realizes it needs a font and when the
necessary text can be rendered with that font. Unlike traditional font subsetting approaches Incremental Font Transfer
retains the encoding of layout rules between segments ([[PFE-report#fail-subset]]).
The success of WebFonts is unevenly distributed. This specification allows WebFonts to be used where
slow networks, very large fonts, or complex subsetting requirements currently preclude their use. For
example, even using WOFF 2 [[WOFF2]], fonts for CJK languages can be too large to be practical.
Technical Motivation: Evaluation Report {#evaluation-report}
------------------------------------------------------------
See the Progressive Font Enrichment: Evaluation Report [[PFE-report]] for the investigation which led
to this specification.
Overview {#overview}
--------------------
An <dfn dfn>incremental font</dfn> is a regular [[open-type|OpenType]] font that is reformatted to include incremental functionality,
partly in virtue of two additional [[open-type/otff#table-directory|tables]]. Using these new tables the font can be augmented
(eg. to cover more code points) by loading and applying patches to it.
The IFT technology has four main pieces:
* [[#extending-font-subset]]: provides the algorithm that is used by a client to select and apply patches.
* [[#font-format-extensions]]: defines the new tables which contain a list of patches that are available to be applied to a font.
* [[#font-patch-formats]]: defines three different types of patches that can be used. Two are "generic" binary patches, one is
specific to the font's format for storing glyph data.
* [[#encoding]]: creates the font and associated patches that form an incremental font.
At a high level an [=incremental font=] is used like this:
1. The client downloads an initial font file, which contains some initial subset of data from the full version of the font along with
[[#font-format-extensions|embedded data]] describing the set of [[#font-patch-formats|patches]] which can be used to extend
the font.
2. Based on the content to be rendered, the client [[#extending-font-subset|selects, downloads, and applies]] patches to extend
the font to cover additional characters, layout features, and/or variation space. This step is repeated each time there is new
content.
Creating an Incremental Font {#making-incremental-fonts}
--------------------------------------------------------
It is expected that the most common way to produce an incremental font will be to convert an existing font to use the incremental
encoding defined in this specification. At a high level converting an existing font to be incremental will look like this:
1. Choose the content of the initial [[#font-subset-dfn|subset]], this will be included in the initial font file the client loads
and usually consists of any data from the original font that is expected to always be needed.
2. Choose the patch type or types. Different patch types have different qualities, so different use cases can call for different
patch types, or in some cases a mix of two types.
3. Choose a segmentation of the font. Individual segments will be added to the base subset by the client using patches. Choosing an
appropriate segmentation is one of the most important parts of producing an efficient encoding.
4. Based on these choices, generate a set of patches, where each patch adds the data for a particular segment relative to either
the initial font file or a previous patch.
5. Generate the initial font file including the initial subset and a [[#font-format-extensions|patch mapping]]. This mapping
lists all of the available patch files, the url they reside at, and information on what data the patch will add to the font.
Note: this is a highly simplified description of creating an incremental font, a more in-depth discussion of generating an encoding and
requirements on the encoding can be found in the [[#encoding]] section.
Performance Considerations and the use of Incremental Font Transfer {#performance-considerations}
-------------------------------------------------------------------------------------------------
Using incremental transfer may not always be beneficial, depending on the characteristics of the font,
the network, and the content being rendered. This section provides non-normative guidance to help decide
when incremental transfer should be utilized.
It is common for an incremental font to trigger the loading of multiple patches in parallel. So to maximize performance, when
serving an incremental font it is recommended that an HTTP server which is capable of multiplexing (such as [[rfc9113]] or [[rfc9114]])
is used.
Incrementally loading a font has a fundamental performance trade off versus loading the whole font.
Simplistically, under incremental transfer less bytes may be transferred at the potential cost of
increasing the total number of network requests being made, and/or increased request processing
latency. In general incremental font transfer will be beneficial where the reduction in latency from
sending less bytes outweighs additional latency introduced by the incremental transfer method.
The first factor to consider is the language of the content being rendered. The evaluation report
contains the results of simulating incremental font transfer across three categories of languages
([[PFE-report#langtype]]). See it's conclusions [[PFE-report#conclusions]] for a discussion of the
anticipated performance of incremental font transfer across the language categories.
Next, how much of the font is expected to be needed? If it's expected that most of the font will be
needed to render the content, then incremental font transfer is unlikely to be beneficial. In many cases
however only part of a font is expected to be needed. For example:
* If the font contains support for several languages but a user is expected to only render content
in a subset of those languages.
* If the content being rendered uses a small subset of the total characters in a font. This is
often the case for Chinese, Japanese, Korean, Emoji, and Icon fonts.
* Only a small amount of text is being rendered. For example a font that is only used for a
headline.
An alternative to incremental transfer is to break a font into distinct subsets (typically by script)
and use the unicode range feature of @font-face to load only the subsets needed. However, this can alter
the rendering of some content [[PFE-report#fail-subset]] if there are layout rules between characters in
different subsets. Incremental font transfer does not suffer from this issue as it can encompass the
original font and all of it's layout rules.
### Reducing the Number of Network Requests ### {#reduce-requests}
As discussed in the previous section the most basic implementation of incremental font transfer will
tend to increase the total number of requests made vs traditional font loading. Since each augmentation
will typically require at least one round trip time, performance can be negatively impacted if too many requests
are made. Depending on which patch types are available and how much information is provided in the
[[#font-format-extensions|patch mapping]], a client might preemptively request patches for code points that are not
currently needed, but expected to be needed in the future. Intelligent use of this feature by an
implementation can help reduce the total number of requests being made. The evaluation report explored
this by testing the performance of a basic character frequency based [[PFE-report#codepredict|code point prediction]]
scheme and found it improved overall performance.
Opt-In Mechanism {#opt-in}
==========================
Web pages can choose to opt-in to incremental transfer for a font via the use of a CSS font tech
keyword ([[css-fonts-4#font-tech-definitions]]) inside the ''@font-face'' block. The keyword
<code>incremental</code> is used to indicate the referenced font contains IFT data and should
only be loaded by a user agent which supports incremental font transfer.
<div class=example>
<pre>
@font-face {
font-family: "MyCoolWebFont";
src: url("MyCoolWebFont-Incremental.otf") tech(incremental);
}
</pre>
</div>
<div class=example>
<pre>
@font-face {
font-family: "MyCoolWebFont";
src: url("MyCoolWebFont-Incremental.otf") tech(incremental);
unicode-range: U+0000-00FF;
}
</pre>
</div>
As shown in the second example, [[css-fonts-4#unicode-range-desc|unicode-range]] can be used in conjuction with an IFT font. The
unicode ranges should be set to match the coverage of the fully extended font. This will allow clients to avoid trying to load the IFT font
if the font does not support any code points which are needed.
An alternative approach is to use the [[css-conditional-5|CSS supports mechanism]] which can make selections based on font tech:
<div class=example>
<pre>
@when font-tech(incremental) {
@font-face {
font-family: "MyCoolWebFont";
src: url("MyCoolWebFont-Incremental.otf");
}
}
@else {
@font-face {
font-family: "MyCoolWebFont";
src: url("MyCoolWebFont.otf");
}
}
</pre>
</div>
Note: Each individual <code>@font-face</code> block may or may not opt-in to IFT. This is due to the
variety of ways fonts are used on web pages. Authors have control over which fonts they want to use
this technology with, and which they do not.
Note: the IFT tech keyword can be used in conjunction with other font tech specifiers to perform
font feature selection. For example a <code>@font-face</code> could include two URIs one with
<code>tech(incremental, color-COLRv1)</code> and the other with
<code>tech(incremental, color-COLRv0)</code>.
Offline Usage {#offline-usage}
------------------------------
In some cases a user agent may wish to save a web page for offline use. Saved pages may be viewed while there is no network connection
and thus it won't be possible to request any additional patches referenced by an incremental font. Since it won't be possible extend
incremental fonts if content changes (eg. due to JavaScript execution), the page saving mechanism should fully expand the incremental font
by invoking [$Fully Expand a Font Subset$] and replace references to the incremental font with the fully expanded one.
Definitions {#definitions}
===============================
Font Subset {#font-subset-dfn}
-------------------------------
A <dfn dfn>font subset</dfn> is a modified version of a font file [[!iso14496-22]] that contains only the data
needed to render a subset of:
* the code points,
* [[open-type/featuretags|layout features]],
* and [[open-type/otvaroverview#terminology|design-variation space]].
supported by the original font. When a subsetted font is used to render text using any combination of the subset
code points, [[open-type/featuretags|layout features]], or [[open-type/otvaroverview#terminology|design-variation space]]
it should render identically to the original font. This includes rendering with the use of any optional typographic
features that a renderer may choose to use from the original font, such as hinting instructions. Design variation spaces
are specified using the user-axis scales ([[open-type/otvaroverview#coordinate-scales-and-normalization]]).
A <dfn dfn>font subset definition</dfn> describes the minimum data (code points, layout features,
variation axis space) that a [=font subset=] should support.
Note: For convenience the remainder of this document links to the [[open-type]] specification which is a copy of
[[!iso14496-22]].
Font Patch {#font-patch-definitions}
-------------------------------------
A <dfn dfn>font patch</dfn> is a file which encodes changes to be made to an IFT-encoded font. Patches are used to extend
an existing [=font subset=] and provide expanded coverage.
A <dfn dfn>patch format</dfn> is a specified encoding of changes to be applied relative to a [=font subset=]. A set of
changes encoded according to the format is a [=font patch=]. Each [=patch format=] has an associated
<dfn dfn>patch application algorithm</dfn> which takes a
[=font subset=] and a [=font patch=] encoded in the [=patch format=] as input and outputs an extended
[=font subset=].
Patch Map {#patch-map-dfn}
--------------------------
A <dfn dfn>patch map</dfn> is an [[open-type/otff#table-directory|open type table]] which encodes a collection of mappings from
[=font subset definition|font subset definitions=] to URIs which host [[#font-patch-formats|patches]] that extend the
[=incremental font=]. A [=patch map=] table encodes a list of <dfn dfn>patch map entries</dfn>, where each entry has a key and value.
The key is one or more [=font subset definition=] and the value is a URI, the [[#font-patch-formats]] used by the data at the URI, and
the [[#font-patch-invalidations|compatibility ID]] of the patch map table. More details of the format of patch maps can be found
in [[#font-format-extensions]].
[=patch map entries|Patch Map Entry=] summary:
<table>
<tr><th>Key</th><th>Value</th></tr>
<tr>
<td>
* One or more [=font subset definition=]
</td>
<td>
* Patch URI
* [[#font-patch-formats|Patch Format]]
* [[#font-patch-invalidations|compatibility ID]]
</td>
</tr>
</table>
Explanation of Data Types {#data-types}
---------------------------------------
Encoded data structures in the remainder of this specification are described in terms of the data types defined
in [[open-type/otff#data-types]]. As with the rest of OpenType, all fields use "big-endian" byte ordering.
Extending a Font Subset {#extending-font-subset}
================================================
This section defines the algorithm that a client uses to extend an [=incremental font|incremental=] [=font subset=] to cover additional
code points, layout features and/or design space. It is an iterative algorithm which, repeatedly:
* parses the font subset's patch mappings into a list of available patches.
* checks if any available patches match the content to be rendered.
* selects one available patch, loads it, and then applies it.
This process repeats until no more relevant patches remain. Since a patch application may alter the patch mappings
embedded in the font file, on each iteration the patch map in the current version of the font subset is reparsed to see what patches
remain. Thus the font subset is on each iteration is the source of truth for what patches are available, and fully encapsulates the current
state of the augmentation process.
Patch Invalidations {#font-patch-invalidations}
-----------------------------------------------
The patch mappings embedded in a font subset encode an invalidation mode for each patch. The invalidation mode for a patch
marks which other patches will no longer be valid after the application of that patch. This invalidation mode is used by the
extension algorithm to determine which patches are compatible and influences the order of selection. Patch validity during patch
application is enforced by the compatibility ID from the [[#patch-map-table]]. Every patch has a compatibility ID encoded within it
which needs to match the compatibility ID from the [[#patch-map-table]] which lists that patch.
There are three invalidation modes:
* <dfn dfn>Full Invalidation</dfn>: when this patch is applied all other patches currently listed in the [=font subset=] are invalidated.
The compatibility ID in both the 'IFT ' and 'IFTX' [[#patch-map-table]] will be changed.
* <dfn dfn>Partial Invalidation</dfn>: when this patch is applied all other patches in the same [[#patch-map-table]] will be invalidated.
The compatibility ID of only the [[#patch-map-table]] which contains this patch will be changed.
* <dfn dfn>No Invalidation</dfn>: no other patches will be invalidated by the application of this patch. The compatibility ID of the
'IFT ' and 'IFTX' [[#patch-map-table]] will not change.
The invalidation mode of a specific patch is encoded in its format number, which can be found in [[#font-patch-formats-summary]].
Default Layout Features {#default-layout-features}
--------------------------------------------------
Most text shapers have a set of [[open-type/featuretags|layout features]] which are always enabled and thus always required in an
incrementally loaded font. [[#feature-tag-list]] collects a list of features that at the time of writing are known to be required
by default in common shaper implementations. When forming a [=font subset definition=] as input to the extension algorithm the client
should typically include all features found in [[#feature-tag-list]] in the subset definition. However, in some cases the client might
know that the specific shaper which will be used may not make use of some features in [[#feature-tag-list]] and may
opt to exclude those unused features from the subset definition.
<h3 algorithm id="extend-font-subset">Incremental Font Extension Algorithm</h2>
The following algorithm is used by a client to extend an [=incremental font|incremental=] [=font subset=] to cover additional
code points, layout features and/or design space. This algorithm incrementally selects and applies patches to the font one at a time, loading
them as needed. As an important optimization to minimize network round trips it permits loads to be started for all patches that will eventually
be needed. [[#font-patch-invalidations]] is used to determine which patches loads can be started for. Any patches which match the target subset definition
and will not be invalidated by the next patch to be applied according to [[#font-patch-invalidations]] can be expected to be needed in future iterations.
<dfn abstract-op>Extend an Incremental Font Subset</dfn>
The inputs to this algorithm are:
* <var>font subset</var>: an [=incremental font|incremental=] [=font subset=].
* <var>initial font subset URI</var>: an [[rfc3986#section-4.3|absolute URI]] which identifies the location of the initial incremental
font that <var>font subset</var> was derived from.
* <var>target subset definition</var>: the [=font subset definition=] that the client wants to extend <var>font subset</var> to cover.
The algorithm outputs:
* <var>extended font subset</var>: an extended version of <var>font subset</var>. May or may not be an [=incremental font=].
The algorithm:
1. Set <var>extended font subset</var> to <var>font subset</var>.
2. Load the 'IFT ' and 'IFTX' (if present) mapping [[open-type/otff#table-directory|tables]] from <var>extended font subset</var>. Both
tables are formatted as a [[#patch-map-table]]. Check that they are valid according to the requirements in [[#patch-map-table]]. If
either table is not valid, invoke [$Handle errors$]. If <var>extended font subset</var> does not have an 'IFT ' table, then it is
not an [=incremental font=] and cannot be extended, return <var>extended font subset</var>.
3. If the compatibility ID in 'IFT ' is equal to the compatibility ID in 'IFTX' this is an error, invoke [$Handle errors$].
4. For each of [[open-type/otff#table-directory|tables]] 'IFT ' and 'IFTX' (if present): convert the table into a list of entries by
invoking [$Interpret Format 1 Patch Map$] or [$Interpret Format 2 Patch Map$]. Concatenate the returned entry lists into a single list,
<var>entry list</var>.
5. For each <var>entry</var> in <var>entry list</var> invoke [$Check entry intersection$] with <var>entry</var> and
<var>target subset definition</var> as inputs, if it returns false remove <var>entry</var> from <var>entry list</var>. Additionally remove
any entries in <var>entry list</var> which have a patch URI which was loaded and applied previously during the execution of this algorithm.
6. If <var>entry list</var> is empty, then the extension operation is finished, return <var>extended font subset</var>.
7. Group the entries from <var>entry list</var> into 3 lists based on the invalidation mode of the [[#font-patch-formats-summary|patch format]]
of each entry: <var>full invalidation entry list</var>, <var>partial invalidation entry list</var>, and <var>no invalidation entry list</var>.
8. Pick one <var>entry</var> with the following procedure:
* If <var>full invalidation entry list</var> is not empty then, select exactly one of the contained entries. Follow the criteria in
[[#invalidating-patch-selection]] to select the single entry.
* Otherwise if <var>partial invalidation entry list</var> is not empty then, select exactly one of the contained entries.
Follow the criteria in [[#invalidating-patch-selection]] to select the single entry.
* Otherwise select exactly one of the entries in <var>no invalidation entry list</var>.
The criteria for selecting the single entry is left up to the implementation to decide.
9. Start a load of <var>patch file</var> (if not already previously started) by invoking [$Load patch file$] with the <var>initial font subset URI</var>
as the initial font URI and the <var>entry</var> patch URI as the patch URI. Additionally the client may optionally start loads using the same
procedure for any entries in <var>entry list</var> which will not be [[#font-patch-invalidations|invalidated]] by <var>entry</var>. The total
number of patches that a client can load and apply during a single execution of this algorithm is limited to:
* At most 100 patches which are [=Partial Invalidation=] or [=Full Invalidation=].
* At most 2000 patches of any type.
Can be loaded and applied during a single invocation of this algorithm. If either count has been exceeded this is an error invoke
[$Handle errors$].
10. Once the load for <var>patch file</var> is finished, apply it using the appropriate application algorithm (matching the patches format in
<var>entry</var>) from [[#font-patch-formats]] to apply the <var>patch file</var> using the patch URI and the compatibility id from
<var>entry</var> to <var>extended font subset</var>.
11. Go to step 2.
Note: in step 9 the client may optionally start loads for patches which are not invalidated by the currently selected entry. This is an important
optimization which significantly improves performance by eliminating network round trips where possible by initiating loads for patches that will
be needed in later iterations.
Note: if the only remaining intersecting entries are no invalidation entries the intersection check in step 5 does not need to be repeated on each
iteration since no invalidation patches won't change the list of available patches upon application (other then to remove the just applied patch).
<div class="example">
An example of the execution of the extension algorithm can be found in [[#extension-example]].
</div>
<dfn abstract-op>Check entry intersection</dfn>
The inputs to this algorithm are:
* <var>mapping entry</var>: a [=patch map entries|patch map entry=].
* <var>subset definition</var>: a [=font subset definition=].
The algorithm outputs:
* <var>intersects</var>: true if <var>subset definition</var> intersects <var>mapping entry</var>, otherwise false.
The algorithm:
1. For each subset definition in <var>mapping entry</var> and each set in <var>subset definition</var> (code points, feature tags,
design space) check if the set intersects the corresponding set from the <var>mapping entry</var> subset definition. A set
intersects when:
<table>
<tr>
<th></th><th>subset definition set is empty</th><th>subset definition set is not empty</th>
</tr>
<tr>
<th>mapping entry set is empty</th><td>true</td><td>true</td>
</tr>
<tr>
<th>mapping entry set is not empty</th><td>false</td><td>true if the two sets intersect</td>
</tr>
</table>
When checking design space sets for intersection, they intersect if there is at least one pair of intersecting segments
(tags are equal and the ranges intersect).
2. If all sets checked in step 1 intersect, then return true for <var>intersects</var> otherwise false.
<div class=example>
<table>
<tr><th>mapping entry</th><th>subset definition</th><th>intersects?</th></tr>
<tr>
<td>
```
subset definitions: [
{
code points: {1, 2, 3},
feature tags: {},
design space: {}
},
],
```
</td>
<td>
```
code points: {2},
feature tags: {},
design space: {},
```
</td>
<td>true</td>
</tr>
<tr>
<td>
```
subset definitions: [
{
code points: {1, 2, 3},
feature tags: {},
design space: {}
},
],
```
</td>
<td>
```
code points: {5},
feature tags: {},
design space: {},
```
</td>
<td>false</td>
</tr>
<td>
```
subset definitions: [
{
code points: {1, 2, 3},
feature tags: {},
design space: {}
},
],
```
</td>
<td>
```
code points: {2},
feature tags: {smcp},
design space: {},
```
</td>
<td>true</td>
</tr>
<td>
```
subset definitions: [
{
code points: {1, 2, 3},
feature tags: {},
design space: {}
},
],
```
</td>
<td>
```
code points: {},
feature tags: {smcp},
design space: {},
```
</td>
<td>false</td>
</tr>
<tr>
<td>
```
subset definitions: [
{
code points: {1, 2, 3},
feature tags: {},
design space: {}
},
{
code points: {4, 5, 6},
feature tags: {},
design space: {}
},
],
```
</td>
<td>
```
code points: {2},
feature tags: {},
design space: {},
```
</td>
<td>false</td>
</tr>
<tr>
<td>
```
subset definitions: [
{
code points: {1, 2, 3},
feature tags: {},
design space: {}
},
{
code points: {4, 5, 6},
feature tags: {},
design space: {}
},
],
```
</td>
<td>
```
code points: {2, 6},
feature tags: {},
design space: {},
```
</td>
<td>true</td>
</tr>
</table>
</div>
<dfn abstract-op>Load patch file</dfn>
<!-- TODO: consider requiring HTTPS (or disallowing HTTP specifically if we want to allow file:// and other url types) -->
The inputs to this algorithm are:
* <var>Patch URI</var>: A [[rfc3986#section-4.1|URI Reference]] identifying the patch file to load. As a URI reference this may be a
relative path.
* <var>Initial Font URI</var>: An [[rfc3986#section-4.3|absolute URI]] which identifies the initial incremental font that the
patch URI was derived from.
The algorithm outputs:
* <var>patch file</var>: the content (bytes) identified by <var>Patch URI</var>.
The algorithm:
1. Perform [[rfc3986#section-5|reference resolution]] on <var>Patch URI</var> using <var>Initial Font URI</var> as the base URI to
produce the <var>target URI</var>.
2. Retrieve the contents of <var>target URI</var> using the fetching capabilities of the implementing user agent. For web browsers,
[[fetch]] should be used. When using [[fetch]] a request for patches should use the same CORS settings as the initial request for
the IFT font. This means that for a font loaded via CSS the patch request would follow: [[css-fonts-4#font-fetching-requirements]].
3. Return the retrieved contents as <var>patch file</var>, or an error if the fetch resulted in an error.
<dfn abstract-op>Handle errors</dfn>
If the extending the font subset process has failed with an error then, some of the data within the font may not be fully loaded and as
a result rendering content which relies on the missing data may result in incorrect renderings. The client may choose to continue using
the font, but should only use it for the rendering of code points, features, and design space that are fully loaded according to
[[#ift-font-coverage]]. Rendering of all other content should fallback to a different font following normal client fallback logic.
If the error occurred during [$Load patch file$] then, the client may continue trying to extend the font subset
if there are remaining patches available other than the one(s) that failed to load. In the case of all other errors the client
must not attempt to further extend the font subset.
Selecting Invalidating Patches {#invalidating-patch-selection}
--------------------------------------------------------------
During execution of the [$Extend an Incremental Font Subset$] algorithm in some cases it's necessary to select a single invalidating (full or partial)
patch entry from a list of candidate entries. The selection criteria used has a direct impact on the total number of round trips that will be needed to
perform the extension. Round trips are costly so for maximum performance patches should be selected in a way that minimizes the total number of needed
round trips.
The following selection criteria minimizes round trips and must be used by the client when selecting a single partial or full invalidation patch in step 8
of [$Extend an Incremental Font Subset$]:
1. For each candidate entry compute the set intersection between each subset definition in the entry and the target subset definition. Union
the resulting intersections together into a single subset definition.
2. Find an entry whose intersection subset definition from step 1 is not a strict subset of any other intersection subset definition.
3. Locate any additional entries that are in the same [=patch map=] and have the same intersection as the entry found in step 2. From the this set of
entries (including the step 2 pick) the final selection is the entry which is listed first in the [=patch map=]. For [[#patch-map-format-1]] this is the
entry with the lowest entry index. For [[#patch-map-format-2]] this is the entry that appears first in the [=Mapping Entries/entries=] array.
Note: a fast and efficient way to find an entry which satisfies the criteria for step 2 is to sort the entries (descending) by the size of the code point
set intersection, then the size of feature tag set intersection, and finally the size of the design space intersection. The first entry in this sorting is
guaranteed to not be a strict subset of any other entries, since any strict super sets would have to be at least one item larger. This approach also has
the added benefit that it selects the patch which will add the most data which the client is currently requesting.
Target Subset Definition {#target-subset-definitions}
-----------------------------------------------------
The [$Extend an Incremental Font Subset$] algorithm takes as an input a target subset definition based on some content that the client
wants to render. The client may choose to form one single subset definition for the content as a whole and run the extension algorithm
once. Alternatively, the client may instead break the content up into smaller spans, form a subset definition for each span, and run
the extension algorithm on each of the smaller subset definitions. Either approach will ultimately produce a font which equivalently
renders the overall content as long as:
* Each span of text which generates a subset definition is built from only one or more complete shaping units.
* Where a shaping unit is a span of text which the client will process together as a single unit during
<a href="https://harfbuzz.github.io/what-is-harfbuzz.html#what-is-text-shaping">text shaping</a>.
Determining what Content a Font can Render {#ift-font-coverage}
---------------------------------------------------------------
Given some incremental font (whether the initial font or one that has been partially extended) a client may wish to know what content
that font can render in it's current state. This is of particular importance where the client is looking to determine which portions
of the text to use fallback fonts for.
During fallback processing a client would typically check the font's [[open-type/cmap|cmap]] table to determine which code points are
supported; However, in an IFT font due to the way [[#glyph-keyed]] patches work the [[open-type/cmap|cmap]] table may contain mappings
for code points which do not yet have the corresponding glyph data loaded. As a result the client should not rely solely on the
[[open-type/cmap|cmap]] table to determine code point presence. Instead the following procedure can be used by a client to check what
parts of some content an incremental font can render:
* Split the content up into the shaping units (see [[#target-subset-definitions]]) on which the content will be processed during text
shaping.
* For each shaping unit there are two checks:
* First, if for any code point in the shaping unit there is not a [[open-type/cmap|cmap]] entry for it, or the entry maps to glyph
0 then the incremental font does not fully support rendering the shaping unit.
* Second, compute the corresponding [=font subset definition=] and execute the [$Extend an Incremental Font Subset$] algorithm,
stopping at step 6. If the entry list is not empty then the incremental font does not fully support rendering the shaping unit.
* Any shaping units that passed both checks can be rendered in their entirety with the font.
The client may also wish to know what the font can render at a more granular level than a shaping unit. The following pseudo code
demonstrates a possible method for splitting a shaping unit which failed the above check up into spans which can be rendered using the
incremental font:
<pre highlight="python">
# Returns a list of spans, [start, end] inclusive, within shaping_unit that are
# supported by and can be safely rendered with ift_font.
#
# shaping_unit is an array where each item has a code point, associated list of
# layout features, and the design space point that code point is being rendered with.
def supported_spans(shaping_unit, ift_font):
current_start = current_end = current_subset_def = None
supported_spans = []
i = 0
while i < shaping_unit.length():
if current_subset_def is None:
current_subset_def = SubsetDefinition()
current_start = i
current_end = i
current_subset_def.add(shaping_unit.codepoint_at(i),
shaping_unit.features_at(i),
shaping_unit.design_space_point_at(i))
if supports_subset_def(ift_font, current_subset_def):
i += 1
continue
if current_end > current_start:
supported_spans.append(Span(current_start, current_end - 1))
# i isn't incremented so the current code point can be checked on it's own
# in the next iteration.
else:
i += 1
current_start = current_end = current_subset_def = None
return supported_spans
# Returns true if ift_font has support for rendering content covered by subset_def.
def supports_subset_def(ift_font, subset_def):
# Return true only if both of the following two checks are true:
# - Each code point in subset_def is mapped to a glyph id other than '0' by ift_font's cmap table.
# - After executing the "Extend an Incremental Font Subset" algorithm on ift_font with subset_def and stopping at step 6 the
# entry list is empty.
</pre>
Any text from the shaping unit which is not covered by one of the returned spans is not supported by the incremental font and should
be rendered with a fallback font. Each span should be shaped in isolation (ie. each span becomes a new shaping unit).
Because this method splits a shaping unit up, not all features of the original font, such as multi code point substitutions, may be
present. If the client is correctly following the [$Extend an Incremental Font Subset$] algorithm with a subset definition formed
according to [[#target-subset-definitions]] then the missing data will be loaded and this case will only occur temporarily while the
relevant patch is loading. Once the missing patch arrives and has been applied the rendering of the affected
code points may change as a result of the substitution.
Note: The "supported_spans(...)" check above should not be used to drive incremental font extension. Target subset definitions for full
executions of [$Extend an Incremental Font Subset$] should follow the guidelines in [[#target-subset-definitions]].
<h3 algorithm id="fully-expanding-a-font">Fully Expanding a Font</h3>
This sections defines an algorithm that can be used to transform an incremental font into a fully expanded non-incremental font. This
process loads all available data provided by the incremental font and produces a single static font file that contains no further
patches to be applied.
<!-- TODO: Especially if we do plan to support having multiple invalidating patches that intersect some categories of
parameter but not others, might it be desirable to have some sort of breadcrumb in the map saying "privilege
this one next if you're trying to load the whole font"? Basically a field or a flag saying "this gets you
to the end fastest"? -->
<dfn abstract-op>Fully Expand a Font Subset</dfn>
The inputs to this algorithm are:
* <var>font subset</var>: an [=incremental font|incremental=] [=font subset=].
The algorithm outputs:
* <var>expanded font</var>: an [[open-type]] font that is not incremental.
The algorithm:
1. Invoke [$Extend an Incremental Font Subset$] with <var>font subset</var>. The input target subset definition is a special one which
is considered to intersect all entries in the [$Check entry intersection$] step. Return the resulting font subset as
the <var>expanded font</var>.
Caching Extended Incremental Fonts {#caching-incremental-fonts}
---------------------------------------------------------------
Incremental fonts that have been extended contain all of the state needed to perform any future extension operations according
to the procedures in this section. So if an incremental font needs to be stored or cached for future use by a client it is sufficient to
store only the font binary produced by the most recent application of the extension algorithm. It is not necessary to retain the initial
font or any versions produced by prior extensions.
Extensions to the Font Format {#font-format-extensions}
=======================================================
An [=incremental font=] follows the existing [[open-type|OpenType]] format, but includes two new
[[open-type/otff#table-directory|tables]] identified by the 4-byte tags 'IFT ' and 'IFTX'. These new tables are both
[=patch map|patch maps=]. All incremental fonts must contain the 'IFT ' table. The 'IFTX' table is optional. When both tables are
present, the mapping of the font as a whole is the union of the mappings of the two tables. The two new tables are used only in this
specification and are not being added to the [[open-type|Open-Type]] specification.
Note: allowing the mapping to be split between two distinct tables allows an incremental font to more easily make use of multiple
patch types. For example all patches of one type can be specified in the 'IFT ' table, and all patches of a second type in the
'IFTX' table. Those patches can make updates only to one of the mapping tables and avoid making conflicting updates.
Incremental Font Transfer and Font Compression Formats {#ift-and-compression}
-----------------------------------------------------------------------------
It is common when using fonts on the web to compress them with a compression format such as [[WOFF]] or [[WOFF2]]. Formats such as
these can be used to compress the initial font file used in an incremental font transfer encoding as long as:
1. The bytes of each [[open-type/otff#table-directory|table]] are unmodified by the process of encoding then decoding the font via the
compression format.
2. Since the incremental font transfer extension algorithm ([[#extending-font-subset]]) operates specifically on the uncompressed font
file, the compressed font needs to be decoded before attempting to extend it.
For [[WOFF2]] special care must be taken. If an incremental font will be encoded by WOFF2 for transfer:
1. If the WOFF2 encoding will include a transformed glyf and loca table ([[WOFF2#glyf_table_format]]) then, the incremental
font should not contain [[#table-keyed]] patches which modify either the glyf or loca table. The WOFF2 format does not
guarantee the specific bytes that result from decoding a transformed glyf and loca table. [[#glyph-keyed]] patches may be used
in conjunction with a transformed glyf and loca table.
2. The 'IFT ' and 'IFTX' tables can be processed and brotli encoded by a WOFF2 encoder following the standard process defined in
[[WOFF2#table_format]].
Patch Map Table {#patch-map-table}
----------------------------------
A [=patch map=] is encoded in one of two formats:
* Format 1: a limited, but more compact encoding. It encodes a one-to-one mapping from glyph id to patch URIs. It does
not support [=font subset definitions=] with design space or entries with overlapping subset definitions.
* Format 2: can encode arbitrary mappings including ones with design space or overlapping subset definitions. However, it
is typically less compact than format 1.
Each format defines an algorithm for interpreting bytes encoded with that format to produce the list of entries it represents. The
[$Extend an Incremental Font Subset$] algorithm invokes the interpretation algorithms and operates on the resulting entry list. The
encoded bytes are the source of truth at all times for the patch map. Patch application during subset extension will alter the encoded
bytes of the patch map and as a result the entry list derived from the encoded bytes will change. The extension algorithm reinterprets
the encoded bytes at the start of every iteration to pick up any changes made in the previous iteration.
### Patch Map Table: Format 1 ### {#patch-map-format-1}
<dfn>Format 1 Patch Map</dfn> encoding:
<table>
<tr>
<th>Type</th><th>Name</th><th>Description</th>
</tr>
<tr>
<td>uint8</td>
<td><dfn for="Format 1 Patch Map">format</dfn></td>
<td>Set to 1, identifies this as format 1.</td>
</tr>
<tr>
<td>uint32</td>
<td>reserved</td>
<td>Not used, set to 0.</td>
</tr>
<tr>
<td>uint32</td>
<td><dfn for="Format 1 Patch Map">compatibilityId</dfn>[4]</td>
<td>
Unique ID used to identify patches that are compatible with this font (see [[#font-patch-invalidations]]). The encoder chooses this
value. The encoder should set it to a random value which has not previously been used while encoding the IFT font.
</td>
</tr>
<tr>
<td>uint16</td>
<td><dfn for="Format 1 Patch Map">maxEntryIndex</dfn></td>
<td>The largest entry index encoded in this table.</td>
</tr>
<tr>
<td>uint16</td>
<td><dfn for="Format 1 Patch Map">maxGlyphMapEntryIndex</dfn></td>
<td>The largest [=Glyph Map=] entry index encoded in this table. Must be less than or equal to maxEntryIndex.</td>
</tr>
<tr>
<td>uint24</td>
<td><dfn for="Format 1 Patch Map">glyphCount</dfn></td>
<td>Number of glyphs that mappings are provided for.
Must match the number of glyphs in the the font file.
Note: the number of glyphs in the font is encoded in the font file. At the time of writing, this value is listed in the
[[open-type/maxp|maxp]] table; however, future font format extensions may use alternate tables to encode the value for number of
glyphs.
</tr>
<tr>
<td>Offset32</td>
<td>glyphMapOffset</td>
<td>Offset to a [=Glyph Map=] sub table. Offset is from the start of this table.</td>
</tr>
<tr>
<td>Offset32</td>
<td><dfn for="Format 1 Patch Map">featureMapOffset</dfn></td>
<td>Offset to a [=Feature Map=] sub table. Offset is from the start of this table. May be null (0).</td>
</tr>
<tr>
<td>uint8</td>
<td><dfn for="Format 1 Patch Map">appliedEntriesBitMap</dfn>[(maxEntryIndex + 8)/8]</td>
<td>