Pop-ups

[dauwhe]

There was a discussion of popped-up content at a Publishing Business Group meeting last night. Copied from the minutes:

Popped Up Content

UNKNOWN_SPEAKER: this is good item to discuss
... Liisa, could you fill in about Popped Up Content issue?

Liisamk: Popped Up Content, and the business requirements around it
... is something we can address without addressing the underlying technology from a reading systems perspective
... what kind of content needs to be enlarged, what kind of control do you as a publisher want to tell the reading systems
... Sometimes footnotes are popped up
... that is not done in a way that publishers have control
... you may put in an EPUB type
... but reading systems may put a UI around it
... we would like publishers to have more control; to know where it starts and stops
... have it gracefully appear and be degraded
... not all reading systems support imaging and formatting inside of popups
... a lot of books like children's picture books use pop-ups
... from bus perspective, you want control of sizing of text, that box
... put a background in it so it is readable
... even when laid over some art
... then extrapolate over the visual narratives
... for visual novels, manga, or other places to enlarge something
... if we could talk about what kinds of control we want
... then we could one, inform the reading systems
... and two, see if we can apply technology
... so that the stuff pops up when you want it to
... sometimes there are circular links....creates a bad reading experience

Murata: this is a very interesting opportunity
... first, is this going to be an activity in PubBG or WG?
... do we do requirements first, or do we investigate solutions at the same time?

Liisamk: I think in the BG, what we want to is to talk about the requirements and expectations and then document them
... and with WG we talk about experiences in the marketplace and see if there is tech we have access to or we can build and go from there

Daihei: so first stage is requirements in the PubBG

Liisamk: yes, the business needs around this

Daihei: the PubBG we would want to confirm and quantify the business needs and then relate to the other working group to have this discussion on the technology side of it
... anybody wants to express their thoughts, take advantage of me being in the Japanese friendly time
... Anybody wants to say something in Japanese?
... I will try to then translate into English?
... Feel free to that here now
... Ok, next item is Popped Up Content
... Everybody thinks it's a good idea to explore in the PubBG

Liisamk: I think the next step to think about
... think about whether the things you want to have pop up are not working well; whether you have control and if it's satisfying your needs and your authors'
... order, speed, explore where the commonalities are
... enough resolution, things popping beyond the border of the pages; those kinds of issues

[dauwhe]

Some background: iBooks will automatically pop-up footnotes if they use the proper markup with aside and epub:type.

Of course, pop-ups are common on the web, and they use JavaScript. It seems like the unstated assumption here is that we want declarative markup that would result in pop-ups without any author JS?

[TzviyaSiegman]

a few additional notes:

• the Publishing WG is looking at replacing epub:type because it is not compatible with HTML
• the EPUB spec does not define pop-ups. This is a behavior that some Reading Systems chose to launch, using the epub:type as a trigger.

[laudrain]

In the existing EPUB3 realm, the PBG seeks to clarify business needs from publishers in order to extend this popup behavior to more Reading Systems, for the sake of better user experience and more business success.
As this behavior can be triggered today in some RS using HTML tags plus epub:type, and as many RS don't accept JavaScript, the idea is to move on the declarative way.

In the future realm of Web Publication, epub:type will no longer exists. The PWG will certainly benefit of this business cases overview to propose pure HTML means to achieve this behavior.

[mattgarrish]

In the existing EPUB3 realm,

HTML 5.2 now has a dialog element. Time might be better spent investigating what it can do than reaching for more EPUB hacks.

[laudrain]

Good to know!
But probably not yet implemented in existing EPUB3 Reading Systems.
In the EPUB3 realm, we are dealing with todays issues, like selling tons of EPUB files.

[mattgarrish]

But probably not yet implemented in existing EPUB3 Reading Systems.

Which is more likely to get support, though: a feature built into the web platform or a hack you have to ask every reading system to individually implement?

[iherman]

I am not familiar with the details, but the example on MDN suggests that even with Dialog some piece of Javascript _is_ necessary… I believe that we have to accept the fact that, on long term, a purely, 100% declarative approach will not be usable...

[laudrain]

If I may, I'd say that in the EPUB3 CG, existing features for the running EPUB eco-system are of interest.
I have no problem with long term acceptance that things will be different and that the work we do in the PWG is about using full web techno for future ebook standard.
The PBG is concerned about how the standard we use today and since a while, meaning EPUB3, may benefit from the community to improve the popup behavior of the files that publishers are building.
That's why the PBG chairs started this conversation that Dave kindly import in this issue.

[GarthConboy]

A number of Reading Systems (e.g., iBooks, Google Play) "do the nice thing" (pop-up with the target) with content marked up correctly with epub:type with values noteref, footnote, and endnote. I said "do the nice thing" rather than "do the right thing" -- as this is behavior that may be desired from the Reading System, but not required of the Reading System. I don't think we can/should get into the business of requiring Reading Systems to support any particular affordance with this declarative markup. But, it's great for Publishers (and the BG and CG) to "encourage" Reading System implementors to "do the nice thing" and it's always good to be nice!

[danielweck]

In cases where Reading Systems "interfere" with the layout / rendering of publication documents (for example: injection of Javascript + CSS to implement column or region-based pagination, continuous reading-order scrolling, etc.), authored interactions can degrade / break (for example: ad-hoc "popup notes" mechanism based on absolutely-positioned div etc.), because content creators cannot rely on a predictable presentation context. In that light, EPUB's declarative "popup footnote" authoring feature may not be perfect, but it certainly serves a purpose. It is basically saying: leave it to the RS, it'll figure out how to "do the nice thing" (GarthTM)

In an ideal world, publication authors could roll out their own preferred techniques for "popup footnote" functionality, and it would be accessible (e.g. screen readers, text magnification, colour schemes, line spacing, keyboard interaction, etc.) ... but this is actually quite tricky to implement correctly "on the web" (i.e. vanilla web browsers), let alone in Reading Systems where the outcome is usually less predictable.

PS: in Readium there is an implementation of "EPUB popup footnotes" that uses HTML dialog. However, just like with div elements before, there needs to be additional Javascript to properly implement "modal" rendering (keyboard focus cycling, background mouse / scroll disabling, etc.). dialog is a useful tool in the toolkit, but it does not magically solve problems.

[JayPanoz]

Yeah I’ve made a very quick and dirty test – please find ZIP that you’ll have to re-package to EPUB attached – to check whether <dialog> is already supported in some RSs and it is in apps using the Android/Chrome WebView for instance.

You need JS, that’s true – unlike <details> (MDN doc).

But what Daniel said:

In cases where Reading Systems "interfere" with the layout / rendering of publication documents (for example: injection of Javascript + CSS to implement column or region-based pagination, continuous reading-order scrolling, etc.), authored interactions can degrade / break (for example: ad-hoc "popup notes" mechanism based on absolutely-positioned div etc.), because content creators cannot rely on a predictable presentation context.

Lots of interactions – even when JS is supported –, and modern CSS modules, are limited by pagination because there’s no proper API to do it for instance – not even taking the different ways it’s implemented into account there, nor other compat/interop issues which make scripting/polyfills notoriously difficult/nearly impossible for authors to design and create.

Actually, pagination also impacts <details> since all RSs don’t necessarily listen to/do layout-change (relevant EPUB 3.2 section).

So unfortunate as it is…

It is basically saying: leave it to the RS, it'll figure out how to "do the nice thing" (GarthTM)

I must admit I’d probably be interested in some sort of “Gap analysis” between web APIs/specs and the ebook ecosystem i.e. newer/latest HTML/CSS/etc. that have minor/significant/major issues in Reading Systems today. It’s been massively useful when I was handling i18n for instance (see Analysing support for text layout on the Web), and given the alignement with HTML latest/CSS snapshot, could also serve as some sort of support matrix/documentation for authors.

But that last point is just my 2 cents.

dialog-test.zip

Edit:

v2 of the quick test file with an example spanning multiple pages – believe it or not, works way better out of the box than I would have expected.

dialog-test-v2.zip

[dauwhe]

I vaguely remember all the fun we had trying to get pop-up footnotes to work in the reading systems that supported them, but also being usable in the reading systems that didn't. This was complicated because, for example, ADE did not recognize links to HTML5 elements like aside. I think we ended up both editing the markup and providing a little JS shim.

[JayPanoz]

OK added a v2 of the test file (example spanning multiple pages) and I’d like to add some nuance to my comment → it’s working way better than I expected in RS supporting <dialog>, and that may also save a lot of Reading Systems quite a headache actually, at least with positioning.

Obviously, doesn’t solve all the issues, but it’s also showing how everyone (i.e. authors, reading systems, users, etc.) can benefit from those APIs in the longer term.

[mattgarrish]

dialog is a useful tool in the toolkit, but it does not magically solve problems.

I don't think that's been suggested. What I asked originally is why we aren't investing more efforts in seeing what can be done with dialogs than trying to breathe more life into a technique that has a limited shelf life and not much better support, it seems.

And are we doing people a service by creating features that only exist in the EPUB bubble? While it might seem like an easy path forward, what happens when this content has to be migrated to an EPUB 4, or needs to also be deployed on the web? It feels like we're just kicking the can down the road.

Calling what we have a hack isn't a statement on the work you or anyone else has done implementing the functionality, by the way, but a question of what we're even standardizing? Pop-up footnotes didn't grow out of anything defined in EPUB. What we have are a few ad hoc implementations, so how does one press for more implementations or get consistency? Is someone writing a spec on how to implement these pop-ups? Where is this going, exactly?

[artbyrt]

I truly dislike the RS footnote pop-up behaviour. For all sorts of reasons: styling, accessibility, layout… My solution (until something better comes along) is to use an anchor tag to note region (using either < aside > or < div >) that moves focus to end of html file and uses CSS styling to indicate the function of the text–including a backlink to the main text. The footnote/endnote should ideally not operate in a way the grossly interferes with the main reading flow in particular from a11y viewpoint. ruth tait

…

On Feb 20, 2019, at 5:45 PM, Matt Garrish ***@***.***> wrote: dialog is a useful tool in the toolkit, but it does not magically solve problems. I don't think that's been suggested. What I asked originally is why we aren't investing more efforts in seeing what can be done with dialogs than trying to breathe more life into a technique that has a limited shelf life and not much better support, it seems. And are we doing people a service by creating features that only exist in the EPUB bubble? While it might seem like an easy path forward, what happens when this content has to be migrated to an EPUB 4, or needs to also be deployed on the web? It feels like we're just kicking the can down the road. Calling what we have a hack isn't a statement on the work you or anyone else has done implementing the functionality, by the way, but a question of what we're even standardizing? Pop-up footnotes didn't grow out of anything defined in EPUB. What we have are a few ad hoc implementations, so how does one press for more implementations or get consistency? Is someone writing a spec on how to implement these pop-ups? Where is this going, exactly? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#80 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AMwGgG31wlmCiSmcL8gZLA66qp0Sf20eks5vPdAIgaJpZM4bFZJo>.

[laudrain]

@mattgarrish in EPUB3, I wouldn't say that coding footnotes with appropriate epub:type values, and using some semantic HTML tag like aside is a hack. Both are available from spec.
That some RS do nice thing with this publisher intent is IMHO legitimate.

Consistency is an issue, yes, as on many other existing EPUB3 features implementations that are discussed in this CG.
I explained above the PBG intents. This thread is up to the CG to go somewhere.

[mattgarrish]

I wouldn't say that coding footnotes with appropriate epub:type values, and using some semantic HTML tag like aside is a hack.

No, but it's not a viable technology to leverage if we want greater compatibility with the Web.

As I said above, the hack is the ad hoc nature of the implementations. Every implementation has been done in isolation, or so I can only assume as there isn't a standard for popping out content, or even a suggestion that it be done. How do we go from that to:

... we would like publishers to have more control; to know where it starts and stops
... have it gracefully appear and be degraded
...
... from bus perspective, you want control of sizing of text, that box
... put a background in it so it is readable
... even when laid over some art
... then extrapolate over the visual narratives
... for visual novels, manga, or other places to enlarge something

These are all great ideas, but how is this achieved by simply putting an epub:type attribute on content?

The solution doesn't have to be dialog but my impression is that it's much better baked already for these kinds of use cases. There have also sporadically been calls for a footnote element in HTML. Looking inward for solutions should always be the last resort.

[JayPanoz]

The solution doesn't have to be dialog but my impression is that it's much better baked already for these kinds of use cases. There have also sporadically been calls for a footnote element in HTML. Looking inward for solutions should always be the last resort.

Disclaimer: I’ve been reluctant to post the following because I absolutely don’t want to be drowned into a heated discussion – and boil as a result – but here we go.

I guess that also brings the overarching question: wouldn’t it be useful to embrace the Extensible Web Manifesto, make a list of interesting features/elements, build a common lib so that people can experiment with those features (see custom elements) and iterate over this initial design? That could eventually lead to more solid cases helping standardise a subset of elements in the HTML spec.

Mind you, epub is in a weird place as it’s an additional layer between the browser/rendering engine and authors. So <epub-popup> or <epub-slideshow> are weird in the sense they would be leveraging a lower-level author-exposed extensibility point that would be used by a community.

However.

The RS developer in me finds it way easier to import such a common lib – and adapt it here and there so that it works as expected – than implementing support from scratch. Believe it or not, it requires an awful lot of time to check other implementations and make yours compatible with others. Personally, I could literally add such a lib as a dependency, import it in my codebase, tweak it a little bit, rebuild, deploy and voilà, “added support for a dozen features/widgets” in my log.

The author in me is curious about using such experimental elements and not worrying about rendering + scripting issues. Getting a11y for free, not impacting user settings in negative ways, etc. Basically, all the issues that require a lot of time when your ebook is not a novel.

The observer in me has noticed the web has definitely gone extensible, and soon you’ll be able to register new CSS properties and even paint.

The pessimist in me has noticed that, right now, the only thing preventing authors from implementing most of what’s supported is EPUBCheck validation. But as soon as it’ll be aligned with Web Specs, all hell breaks loose. You could use that to prevent user settings from being applied, construct your own elements in a non-accessible way, etc.

So the optimist in me thinks that maybe, maybe if some effort is put into such R&D, it will 1. empower authors and 2. close the gap with Reading Systems – by enabling new authoring options that will make awful CSS/JS hacks unnecessary.

But maybe that’s just me, in which case ignore that comment – not a hill I want to die on.

[dauwhe]

@liisamk, I think a good way to move forward would be to build the user experience you want using script, essentially polyfilling what you want. Perhaps this could take the form of some custom elements for the various use cases. Sounds like a really fun project for your team!

[dauwhe]

Some comments from Baldur on Twitter: https://twitter.com/fakebaldur/status/1099001578051305473
I think he has a good point about the footnotes UI. And I worry about the accessibility of all of this.

[JayPanoz]

@dauwhe yeah I was just about to follow-up after this twitter discussion, to clarify my previous comment.

I’m working for a company whose background is analytics so we’re very open to A/B testing. But if say feature implementation B performs way better than feature implementation A, no matter the lib says feature implementation A should be enforced, feature implementation B would be the one ultimately implemented after consulting all involved parties.

eBook UX is another pet peeve of mine actually, because we need research/data backing it up. But it’s been lacking. Dramatically. And we must therefore manage it.

That should echo the “doing the nice/right thing” point in this discussion.

[JayPanoz]

And I worry about the accessibility of all of this.

Last time I checked, I think at least one fruity Reading System’s popup footnotes weren’t necessarily – I couldn’t tab-navigate it at least, and failed to make the screen reader read them consequently.

Since I’m not a regular VoiceOver user, I told myself that maybe I was missing something and decided to “play safe” and mark up footnotes so that they are displayed @ the end of the chapter instead of being hidden.

[dauwhe]

1. dialog is implemented by only one browser.

2. it requires JS to work at all.

3. there seem to be major accessibility issues with dialog, which the one implementor is not interested in fixing. See https://www.scottohara.me/blog/2019/03/05/open-dialog.html

[danielweck]

there seem to be major accessibility issues with dialog

Sadly, for the important technical parts of a "modal dialog" UX, the HTML dialog element is no better / no worse than prior div-based art: a non-trivial amount of Javascript and CSS is still required to make a "modal" dialog actually modal (i.e. from a keyboard perspective and mouse/pointing device interaction => cyclic focus trap, control autofocus on open/close, background anti-scroll, click-to-dismiss, etc. etc.)

I've actually used dialog in a Chromium-based app, with all the additional "bits" that make it truly modal and accessible ... however the net benefit compared with other div -based approaches can probably be summed-up in 2 bullet points (from the top of my head: .showModal() API, and native support for the notion of a "background" overlay?).

[RachelComerford]

This was an interesting conversation but is out of scope for the EPUB CG to resolve. If it's still of interest, it could be taken to the newly formed Publishing CG for incubation.