Sunday, April 25, 2010

Extending web browser - HTML 3D/stereo support

Extending web browser - HTML 3D/stereo support

Article is under construction

Any comments and suggestions are appreciated.

Brief

Current browsers do not have much of stereo media support. With relatively small changes in CSS and rendering engine open source browsers could become stereo-capable. Accounting appearance of 3D TV sets and channels and embedding browsers in settop boxes ( not to mention 3D game consoles), HTML standards shall be extended for new media vision. Still having 2D engine to render stereo content.

Intro

While been working on speed analyzing on embedded browser, I realized that change of the browser side is unavoidable to keep the HTML logic relevant to developer's needs as from performance as from API use convenience, not to mention web application design and architecture. Having fear of breaking standard behind, I started to think about web browser evolution.

 3D which was so attractive in the past, finally came in our lives. With enormous success of Avatar 3D movie, entertainment industry has been pushed towards 3D. And that is not just on movie theater screen. In some developed countries 3D TV channel are available, you could buy 3D BlueRay  disks and players, Stereo-capable TV sets. TV and settop boxes are capable of web browsing and most of them have UI based on HTML browser. All of listed are prerequisites of taking advantage of 3D in HTML browser. It is only question of time which device will extend HTML to support 3D-like UI.

What to expect

The first thing stereo-capable device browsers will adopt is HTML controls stereo-skinning: from borders and scroll bars to buttons. And quite fancy at the moment controls made by flat images on those devices will  look as stone-age drawings.

After specific devices and following their popularity the frameworks will adopt stereo or even 3D UI. Flash, SilverLight, Java UI are on the list. After Quake was played in browser I will be not surprised if JS libraries will offer set of stereo themes.

I have no doubts of eventual acceptance of 3D with model similar to OpenGL on the web. But it will not replace the HTML as primary structural container textual and media content, including some of 3D. Instead it will be embedded into HTML as plugin or namespace similar to 2D vector graphics (SVG/Canvas). Unlike full-pledged 3D, support for stereo features on HTML itself is on demand now.

Stereo vs 3D

Even if the single human eye is capable of depth perception, the major 3D impression comes from stereo vision. When each eye is seeing the own  perspective. The depth details come from the differencein each eye picture. The true 3D at the moment had not come to our computer screens yet. We need to live with 3D surrogate named stereo.

Games developers invented another substitution for 3D which stays between stereocouple and actual 3D: layered in z-index plains. In CSS z-index serves just a purpose of what layer will be rendered on top. It does not reflect the 3D depth.

Having the stereo output does not require having 3D engine on browser side. The stereo support in browser could be substituted with preparing of stereoimage couple by UI designers. In most cases the difference between two images will be in shift in shadows or HTML components. But how big the difference (shifts) shall depend of personal criteria and device use cases. Having on hands 3D scene parameters for HTML elements like x, y, depth and orthogonal vector will give enough input to 3D rendering engine, but too much on 2D one.

Having 2D engine and using additional depth coordinate along with scene parameters could give good stereocouple rendering capabilities.

While the top,left,depth belong to the HTML, the scene view parameters are device and even viewer-specific. Scene parameters could be switched or even dynamically changed over time. For example, following viewer(s) eyes while moving across the room or distance to device changing.

Stereo HTML vs 3D, evolution vs revolution

The earlier attempts to introduce 3D into web browser successfully failed. VRML, O3D and zillion of others. IMHO, the reason is that web needs do not match 3D engine. HTML serves the different stuff than 3D engine. The target is structured text, images and other embedded content. That is a container logic with some important content blended in: text, CSS, JS.  For some reason only JS and embedded objects(plugins) had been covered to some extent. Due to separation of plugins from browser, all of them are still counted as proprietary. The most popular at the moment - Flash - still approaching the standard level sneaking into browser. Not to mention on W3C level.

Leveraging of existing HTML/web industry standards, tools and trained labor force will give HTML-blended solution huge advantage over independent 3D namespace or language.

From amount of effort on web browser modification the stereocouple rendered by slightly modified old 2D engine wins over real 3D support.

CSS media

The basic operations for stereo couple rendering on HTML elements will be shadows and elements shifts. Those shifts relatively easy could be done in CSS. Once CSS is aware of left/right (or middle:) eye media. The media is most appropriate CSS entity to be used as selector part for projection (left/right eye image) separation.

Some samples of media selectors:

  • screen(default - middle). Current flat plain 2D presentation. Screens of many kind - from widescreen TV to portrait iPhone..
  • left and right stereocouple
  • 3D printer. Kind of funny that same HTML could be used for printing not just on regular but 3D printer. Logic allows that. As result - completely new use case.

Rendering engine

For the stereo presentation the browser will need to render separate image for each side based on scene parameters and stereocouple CSS.

In the first cut scene parameters could be skipped for simplicity. They will use defaults for device. Which will leave the same HTML rendering engine with extra filter on top to keep only stereocouple left or right part.

3D HTML support

Obviously some extra services could be given by engine approximating the use to 3D. For example, extending HTML z-index model with depth, so each element will be granted the volume. The light source driven by user settings will make shadows and sparkles on the buttons and other elements. Borders instead of plain width  will accept the radius in similar to corner radius. And so on.

stereo xhtml namespace

In the situation when the stereo web page need to be backwards HTML compatible the xml namespace is ideal way to extend HTML syntax. Stereo namespace will be responsible for stereo features blended into HTML to extend existing syntax

3D xhtml namespace

Unlike stereo namespace the 3D namespace will be matching the need of 3D engine. It will interact not just with HTML, but with animation, physics and so on. The closest match will be VRML.

Js3D and xhtml namespace

Current 3D engines like O3D are exposed to HTML only or mostly via JS API. But most of such functionality could be presented as XML and blended into XHTML. That way integration will be  more tight and native. The JS is good to some extend to present the 3D model behavior and OK for creating scenes. But declarative XML will serve better for 3D scene definition. Even now the scene primarily imported before been used by JS, why not to formalize it along with declaration of JS API? In fact to have a complete solution, declarative approach shall lead the API one.

Links

Keywords

Support of stereo in HTML. 3D in Screw the w3c. w3c suck. Make own browser. Extend the browser.

2010/04/25

©2010 Sasha Firsov

5 comments:

  1. In regards of screen stereo support, most attractive is having media PC connected to two LCD matrix based polarized projectors. That combination given static image on each eye, biggest screen size, no-weight polarized glasses. It is costly but somehow real.

    As far as I know at the moment no other technology given better combination.

    All monitor based solutions are suffering from blinking (frame-per-eye) and heavier glasses with electronic inside. Claim is that on 100Gz eyes do not fill blinking. Still looses to static image though. But is only realistic solution for gamers:
    http://3dvision-blog.com/tag/3d-photo-in-browser/

    Color filters obviously are not best ones either. Not suitable for daltonics. And normal eyes have a risk to get that desiese over time.

    Wearable displays have a good potential. But have you seen any 1080p ones? It is kind of strange why this promising technology did not come to standard HDTV resolution. May be due to good military applicability?

    ReplyDelete
  2. I think that there is no a software problem to add 3D to a browser, the problem is that current 3D devices are really cumbersome and inconvenient. When the hardware catches up next day there will be a 3D interface, next year a really useful 3D interface.

    ReplyDelete
  3. Anonymous27/1/11 00:02

    @Sasha see the Zalman monitor. Uses circular polarised filters.

    ReplyDelete
  4. @Anonymous A) "due to the Polarized stereoscopic view you are really looking at 1920x540 pixels in total" Which mean "interlacing" for each eye is in place.

    B) Zalman monitor(ZM-M240W) at the moment uses linear polarization.

    That said the quality still quite behind of 1080P.
    http://www.zalman.com/ENG/3D/skill01.asp

    ReplyDelete
  5. Sony is releasing the HMZT1 Personal 3D Viewer. 720P head mounted display. http://bit.ly/oOIbSh

    ReplyDelete