-
Notifications
You must be signed in to change notification settings - Fork 60
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
iframes and external content #1061
Comments
Some few further details. In practice Reading Systems might be wary of handling exceptions like Youtube/Vimeo/etc., given the current handling methods, which vary from:
I guess 1. and 2. for security (and political) reasons. |
Personally, I'd like to see external content beyond video streaming allowed, such as interactive HTML widgets. Some widgets that can work locally can already be supported in FL EPUB3 but there would be great value to see support for streaming IN content from outside of the book (so quiz content can be kept topical) and also to allow to post OUT to a reporting system (so quiz answers can be analysed). |
Should it fall into Remote Resources (or any problem with treating iframe as Remote Resources)?
|
Made a very minimal sample EPUB with an iframe pointing to an external website, and tried it with some reading systems:
|
Pretty hard to offline this content too. I don't think we should bite this off for 3.2. |
CG resolved to defer this to the next version of EPUB. The implications for the user experience, security, IP, etc. are profound. We need to get all our 🦆s in a row. |
From web browser point of view I know iframe and a (href to outside) are different, but from internal resource or external resource point of view (which reading system more care about) I can not see much difference between iframe and a (href to external resource of epub) Also, offline reading (which is one of major use case of epub) is one of my biggest concern. As reading system we can not cache or prefetch external resource and also need to let user know exactly some specific part of content can not be displayed is because of they are offline. |
To be clear, what we are saying here is that in a timeframe where we are trying to encourage greater and greater adoption of EPUB, and the tools used to create it, if someone (say, a teacher creating a course syllabus) inserts a link to a youtube video, our stance is that should not be allowed in the spec, and should fail EPUBcheck, and not be encouraged or even supported by a reading system? |
That should fail epubcheck, as it being a foreign resource isn't the primary problem but that you're not allowed to reference resources outside the container except in a few cases:
|
There is a danger that we throw the format out the window to let html content in (rules, meh, I'll just iframe it!). But in the interests of happy mediums, is there any way to find consensus around this as a remote video issue? I don't think the specification should dive into whitelisting video sites, but what if we had a statement like:
And to that add an additional clarification that it is up to reading system vendors to determine what sites they will allow references to, if any. Probably in the section I cited above. A toe in the door kind of approach, maybe? |
@mattgarrish see #252. I think we have some contradictory language in our specs :) (GitHub doesn't allow EPUBs as attachements; changed extension to ZIP) |
Not sure I'm following. A foreign resource isn't a remote resource, it's just one that isn't a core media type. You need to provide a fallback for foreign resources from an iframe using maifest fallbacks, yes, but that doesn't mean you can use an iframe to reference web-hosted content. |
Looks like epubcheck is wrong here? |
Deleted my example above. It appears to be a bug in epubcheck, and confusion between "foreign" and "remote." |
An additional datapoint for @dauwhe's list. Tried his sample in the Readium Chrome app and it failed with these errors:
This isn't too surprising as Chrome is rather restrictive about security rules with respect to Chrome Extensions and Apps. Alternatively, I tried the sample in the Readium CloudReader (0.31-alpha) and it appears to work fine. |
Filed a bug on epubcheck w3c/epubcheck#852 This is both an interesting hack, and a possible way forward. In general EPUB uses fallbacks when content might not be available to a particular user. This is a similar situation, except rather than being a different media type the resource is remote. |
The issue was discussed in a meeting on 2020-12-11)
View the transcript1. External Content and iFramesSee github issue #1061.. Wendy Reid: the use case that brought this up was linking YouTube videos (or similar content) in textbooks. Dave Cramer: this is a complicated issue because it pulls us in 2 different directions. Marisa DeMeglio: Has anyone given much thought to the relationship between fallbacks and external content?. Dave Cramer: one reason that we put this on the agenda was to get a sense of whether this use case is important in the Asian market. Garth Conboy: how far along are we to allowing video content without fallbacks? Are we already there?. Wendy Reid: in practice, i've seen video elements with external urls as href. Brady Duga: it feels like this is worse, because video and audio are limited in what you can do with them. Dave Cramer: yeah, it's a slippery slope. Wendy Reid: i'm pretty sure that in the wild i've seen youtube videos in ebooks (not that they work well/at all depending on the RS). Brady Duga: we'd like to think that epubs are essentially web content, but there are lots of ways in which RS and browsers are different (e.g. security). Ben Schroeter: i think the fact that we've already seen this in the wild pushes me to err in favor of wider adoption, but also mitigate it in some way. Brady Duga: I'm not seeing youtube in ebooks.
Wendy Reid: the place i think it might happen more often is where size limitation come into play. Ben Schroeter: I do see how it is a slippery slope, we should try to make that slope less slippery. Dave Cramer: there are a lot of moving pieces here... maybe some of us should revisit the issues in more detail, get the problem in better focus. Brady Duga: I'm thinking about the security implications of this, and we aren't ready for this. Wendy Reid: one of the things we should start thinking about is horizontal review. Dave Cramer: it sounds like we're not ready to resolve on this right now. Wendy Reid: agree. Dave Cramer: so we take it back to github for now. Wendy Reid: might be worth taking this to the business group? Asking e.g. if there is a need to make special provision for video in addition to what we already have..
|
I am a supporter of allowing External Content in EPUB. Maybe we should have a way to declare to the reading system, from within the EPUB metadata, which external links we want to allow in the page content? A reading system could then choose how to flag this potential security risk to the user and give options whether to allow this external content? Maybe based on an allowed or disallowed list? There are great things that an offline EPUB will never be able to achieve but I would argue should be supported and allowed within EPUB. Here is a video clip of Sketchfab 3D model working in a fixed layout EPUB in Thorium. https://www.youtube.com/watch?v=P_PMLeqbbjQ Here is video clip of legally streaming original music from Spotify into an EPUB using Readium on the web. https://youtu.be/0j-HC_4M2sA?t=932 Also the communication from a book out from the EPUB to a cloud based service has great potential and I would recommend this is also be supported or at least not disallowed. Here is a video clip of BookWidgets student quiz results being sent to a marking system from Thorium https://www.youtube.com/watch?v=x9Np1CDjLFI Looking up this thread I was pleased to see I said something similar on 5 Jul 2018 and I still agree with myself. |
The issue was discussed in a meeting on 2021-05-27
View the transcript4. iframes and external contentSee github issue #1061. Dave Cramer: example use case would be embedding youtube video in iframe Ben Schroeter: I think publishers may be doing it already Dave Cramer: I think there is a fairly high amount of epubs with video content in higher learning books Brady Duga: but are those videos embedded in the epub, or a link to a youtube video that is supposed to open in a player? Shinya Takami (高見真也): the epub spec should allow this external content, and RS should take care of making associated alerts to users Dave Cramer: that puts a lot of burden on RS to evaluate every URL of this nature and try to figure out if it is safe or not Shinya Takami (高見真也): the alert could just tell users that they are about to access a foreign resource, and that it may be dangerous Brady Duga: users might be scared away by such alerts Wendy Reid: agreed Shinya Takami (高見真也): how about adding features to RS to allow user to toggle whether to permit or deny things like foreign resources, scripting? Wendy Reid: i think that's outside of scope. We can't tell RS how to handle privacy. Brady Duga: it also assumes that the user is legally able to give consent to whatever might happen Dave Cramer: in some circles i've heard concern over how web handles this sort of issue today (e.g. ubiquitous cookie consent pop-ups) |
The issue was discussed in a meeting on 2021-10-08
View the transcript2. iframes and external content (issue epub-specs#1061)See github issue epub-specs#1061. Wendy Reid: this issue is from epub 3.2 and we also discussed at last F2F. Last time we said that there are use cases for external content in epub, BUT its not uncommon for RS to block this content for security and privacy concerns. AND its not up to us to tell RS how to handle these security and privacy issues, which leads to usability and interop issues. George Kerscher: it would be great if publisher could put in picture of something classically in textbook, and alternatively link out to a youtube video of same
Ivan Herman: I want to understand what are our reasonable options here? Dan Lazin: first, I think it would be great to support it. This is more secure than arbitrary js inside the epub Brady Duga: I think this is a more specific question. We recommend container constrained scripting. In that that case the script would have to be part of the epub, which is okay, since we want the epub to be self-contained Wendy Reid: and even where you do host the content, but the ebook is sold via an external retailer, then you have the same security issue arise Ivan Herman: the biggest argument raised against allowing this is that ebooks no longer become readable offline Brady Duga: the security issues around this aren't clear. We assume RS is a web browser, but it often isn't. It's usually a webview, which lacks the security model that a browser has. Hadrien Gardeur: this is all very confusing. Why are we talking about how something should be implemented and iframes? Ivan Herman: the question is whether external iframes should be allowed or not Wendy Reid: as ivan said, this isn't addressed in the spec. Addressing it could involve some kind of normative statement about iframes, which creates a testing problem where RS have arbitrarily determined how to handle this Hadrien Gardeur: as a RS developer, iframes in the content are you enemy. Iframes in your RS are a different thing. But iframes in content are always a problem. Wendy Reid: so even if we said something about it, in the real world we're going to see a variety of different results Ivan Herman: what happens if epub spec forbids use of iframes in content? Is this the kind of statement you would like to see? Hadrien Gardeur: as RS developer it would help, yes Ivan Herman: I don't know if the community would accept such a normative statement, but its a possible direction George Kerscher: why are we forcing this into an iframe and only an iframe? Matt Garrish: I think its weird to get into banning HTML elements. Iframes are useful and have been used in existing content. Brady Duga: we do allow external audio and video, and typically you'll get a play button in your book. The reason iframe is interesting is that its not necessarily a video resource. Its an entire page inside the book that could contain its own scripting, etc.
Hadrien Gardeur: RS can recognize templates for link and provide special handling if they want, but that's outside of spec
Wendy Reid: sorry to all queued members, we're out of time |
Which demands with this issue? (a) to use the video from foreign resource If (a) , no need to use iframe. YouTube URLs are not immutable. The URL changes if the video is updated.
If it's a website, we can fix it quickly, but it's difficult to fix it with EPUB that has already been distributed. |
The question seems to be, is an iframe which references a non-local URL allowed per the spec. Resource Locations seems to address this: https://w3c.github.io/epub-specs/epub33/core/#sec-resource-locations Reading that seems to say no, this is not allowed, since the resource in question is not an audio resource, video resource, or font, nor is it being loaded by a script (though it may be loading scripts). The only argument I can see to allow it is to claim that an embedded video player is a video resource, since it plays a video. Our definition of video resource is sadly unspecific since we don't restrict codecs, but I believe the assumption is that a video resource should be some sort of direct media container and not a html document, script, or other non-video file. There may be a desire to allow this, but that seems like a feature request out of scope for epub 3.3. I would propose closing this, although we may want to consider adding language either to the definition of video resource to clarify what we allow as one, or clarify the Resource Locations section to point out that iframe is not a loophole to allow loading of arbitrary external resources. |
Just to be clear, in current distribution systems, large EPUBs are not allowed, and thus embedded video is not being done. Add to this that two codecs are embedded, because we do not require one video type, makes it worse.
Agree that it may be out of scope for 3.3, but we need to get it in the feature requests for the future. While we are at it, we need to make sure transcripts and the audio description track for videos that are described are supported.
|
I agree that allowing large resources like audio and video to be external to the EPUB itself (that is, not part of the zip package) is a good idea, but that is allowed today. But the list of external resources is limited to audio, video, and fonts, plus some carve outs for scripting which is another topic. In the example provided, a video resource is not being referenced, an external web page is. There is no provision for that in the spec, as there really is no size argument to made for html content. Of course, you can always provide a link to external content like a video sharing website, but that will not be embedded in the content. It will likely open a new window, tab, or app depending on various device conditions. Given that, what feature are you requesting? We could open all resource types to external content, though at that point I am not really sure why you wouldn't take the next logically step and make your book a website. Still that seems to be an entirely different question and should get a new feature request. |
Just to clarify... I believe the original issue is really on youtube or vimeo (or similar) videos (or media) only, where the available URL-s are not referring to the media files themselves. I have just tried a simple HTML file with the URL cites by the original question (i.e., I agree with @toshiakikoike that including a reference, to be very specific, to a YouTube video (via iframe) may not be a good idea for an offline-able medium like an EPUB in the first place. However:
If none of these examples really rely on iframes, and remembering @HadrienGardeur's reaction on the call whereby "as a RS developer, iframes in the content are your enemy", then I would agree to postpone this issue for now and not to extend version 3.3. |
The examples shown in my #1061 (comment) have been placed in iframes but are NOT videos. I was showing some examples of interactive online content as this is a thread is about external content. The first two examples can only be achieved when online (interacting with online 3D models and legally streaming online music). The third example (an interactive quiz) is extended by communication out from the EPUB to an online education service. I can give other examples where iframes can be used to embed local but independent, already-existing HTML content into an EPUB page such as this https://www.youtube.com/watch?v=hPMxIj6-kyQ To me, enhancing an ebook with curated content from a specific resource to the reader (students or otherwise) alongside regular content on the same page is very powerful and very different to suggesting they go away and open a link in a browser or that the publisher should attempt to 'make the whole book into a website’! Reading systems might choose to disallow external content in iframes or even all iframe content, on security or complexity grounds, or they may wish to alert the reader to make that choice. Maybe we should insist on a placeholder or fallback be added to the EPUB for these cases? I encourage you not to disallow either iframes or external content for current and future EPUBs and reading systems that do use and support it. |
I'm not doubting the usefulness of being able to embed external content, but allowing it blurs the lines between reading a book and surfing the web. How does a user know which they are doing as they read? More to the point, what safeguards are there for users who may not want their information sent off to a third party api, and may not realize that's what's happening when they submit a form or interact with a widget? Does the author have to sanbox the iframe to prevent malicious code from interacting with the book, or are we expecting reading systems to add this? I agree with Brady that if we're going to add this as a feature we at least need more time to work out the implications of it. |
In my simple use case of a publisher wanting to use a video as a substitute for a still image, I do see where the publisher hosting the content on their servers where such assets are stored for EPUB (and HTML) usage is better than a YouTube example. I don’t think there is an iframe requirement, the publisher just wants to know the best practice technique to do this.
This is brought around by distribution limitations of size.
|
Looking into the issue a bit further, also in line of newer comments... As @bduga said in #1061 (comment), the original request cannot be fulfilled per spec today because it says:
However... the problem I see is that this is not a checkable requirement. The only way a tool like epubcheck could find out whether a URL points to one of the permitted resources is to make an HTTP request for the head and look at the media type. I do not think it is realistic to expect a tool to do that (e.g., it would preclude off-line usage of epubcheck). What this means for me is that, if we want a spec that reflects the current situation, the MUST above should actually be a SHOULD. But then of course the situation becomes different. The current state is that iframes with external references already work for some RS-s as of today, as shows in the early tests of Dave or of Ken (the latter shows a book widget running in Thorium). And, with a SHOULD, there is no reason for an author not to do this. I.e., the current and, probably, purely aspirational restriction in the spec has already been bypassed in reality. So maybe the right approach is to say:
|
I'm not sure we want to use epubcheck as a design basis for the format. 😄 If we go that route, it calls into question most of the content documents restrictions. epubcheck doesn't load documents, either, so it really has no way of knowing what the dom actually consists of after all the scripts have run. That's always been a shortcoming you could use to work around this issue rather than simply lie about the media type in the package document (i.e., rewrite the src attribute of the iframe after the page loads). Assuming we don't loosen the restriction, though, there probably are ways we could beef up validation. The simplest one might be to specify where remote resources are allowed, not just what types. So the wording, for example, could be "Audio resources when referenced from the That would allow epubcheck to more easily flag any remote resource referenced where it's not allowed, as it wouldn't have to know anything about its media type. It would also limit potential abuse, as you can't load a random resource into an |
The issue was discussed in a meeting on 2021-10-14 List of resolutions:
View the transcript2. iframes and external content (issue epub-specs#1061)See github issue epub-specs#1061. Dave Cramer: this is the issue of using iframes to bring in external content (e.g. youtube videos) into epub
Dave Cramer: there's been fair amount of discussion on github since the last meeting Brady Duga: we need to focus on the issue, which is whether it is legal to do this in the spec. Answer seems to be no.
Dave Cramer: last month we had our first meeting with PING about epub Matt Garrish: ditto, this hits on issues across the board related to horizontal review. Even a11y. Dave Cramer: you were saying you could only use remote video via the Matt Garrish: right, and I'm sure most people use remote resources this way anyway Brady Duga: does Matt Garrish: does Brady Duga: I haven't seen it used that way, just a weird edge case Dave Cramer: Brady Duga: I don't have a problem with limiting external videos to the Shinya Takami (高見真也): i wouldn't recommend opening the spec to enhancements in external resources now Dave Cramer: seems like we're talking about a feature request, in which case we could defer this to a future revision
Brady Duga: it's a little weird to turn this into a feature request, because the original issue was just asking for clarification on whether or not it was legal to do the youtube embedding thing
Dave Cramer: the comments kind of made it into a feature request
|
I have created an enhanced epub with five videos embedded in the iframe, and the book works fine for a few readers. However, when I run epubcheck, it gives me an error as epub standards are set not to accept iframe. Many suggest using |
From RickJ:
The text was updated successfully, but these errors were encountered: