Lately we’ve been working with Unity to produce interactive visualizations of 3D voxel data. We’re developing a pipeline that can quickly render a dataset as a rotational image stack and a front-end application that displays it as an interactive turntable. Unity’s ability to compile for smartphones, websites and standalone platforms has us dreaming up all kinds of uses for this technology.
Notes:
- You need the free, lightweight Unity web player plugin available here.
- The Lazy Susan download is about 4MB.
- Click and drag horizontally to rotate the chameleon. Dragging vertically will blend between the skin and skeleton of the beast.
- If you have an old graphics card, you might not see anything below. Let us know in the comments if this happens to you!
May 18th, 2011 at 8:18 am
Awesome work guys! Love the vertical functionality!
May 18th, 2011 at 10:25 am
is there a chance to see “Beauty” instead of “Beast” ?
May 18th, 2011 at 10:38 am
Guys, please, can you force me to some info about that technology?
How can i icheve something like this? Or i can`t do that with available tools for fusion ?
Actually yours beast gives me some ideas – where i can use such representation of data in our work.
thanx!
May 19th, 2011 at 10:38 am
Hey russiaman,
It would be hard to cover volume rendering in a blog comment, but the key to our Unity demo is that any sequence of images can be turned into a Lazy Susan by importing it as a large tiled texture. We used our Quilt tool (see “Tools” above) to generate two of these textures in Fusion, which Unity consumes and blends between. –Matt
July 12th, 2011 at 11:54 pm
Hello, Matt,
I’ve used interactive visualization of 3D voxel data in desktop applications using OpenGl 3D textures, but when I started to import the project in Unity3D I disappointed that it does not support ones.
Would you tell me, please, which approach you use to visualize a large tiled texture, is it a particle system with script-positioned and coloured particles, stack of 2D texture slices or some else?
I’m also interested how to make the visualisation interactive (cutting planes, voxel filtering, 360 rotation etc.) and how to increase the quality of a low-resolution textures using interpolation techniques (for ex. like Klaus Englel’s pre-integrated volume rendering http://wwwvis.informatik.uni-stuttgart.de/~engel/pre-integrated/). Is it possible in the Unity?
Thank you for your help, details and sources will really appreciated
-Ruslan
July 14th, 2011 at 8:59 am
Yes, our attempts at doing volume rendering in Unity hit the same odd barriers. Unity artificially limits what we can do with the hardware, both through a lack of 3D textures (in OpenGL or Direct3D) as well as a limit on size/complexity of shaders. SM3 is over 7 years old, too. It’s pretty frustrating.
Pre-integration (I couldn’t open your link, but I think I remember the paper) doesn’t help a low resolution dataset, it helps a low sampling rate render. You need two shaders, one that processes your transfer function to generate a new lookup (with an increase in dimentionality, so a 1D LUT becomes a 2D one) and another that uses that new LUT to determine a likely value for the result based on a pair of samples (like along a ray). We’ve tried this in Fusion, but I don’t think we did in Unity, though we probably should.
July 14th, 2011 at 11:47 am
vrusua,
We did write a Unity shader that consumes a volume dataset as a large tiled texture displayed on a “stack of planes”, but the performance doesn’t seem good enough for the typical web user. Modifying this shader to do ray marching from a single plane improved performance significantly, but we ran into the Unity SM3 instruction limit because of the unrolled loops. We may try this again for a small dataset and post the results here.
What you’re seeing in this example isn’t volume rendering on the client machine, but is actually a little trickery with blending between two pre-rendered image sequences. The challenge for us now is to design an efficient pipeline from the starting dataset to reach the result you see above.
July 14th, 2011 at 12:29 pm
Thanks Chad,
I just corrected the link of the Klaus Engel’s work:
http://wwwvis.informatik.uni-stuttgart.de/~engel/pre-integrated/
His method is based on the rendering of additional interpolated slices, voxel values of which (color and opacity) are calculated (pre-integrated to the lookup table) based on voxels values of neighbour slices from the original dataset. So it can increase the “image quality even for low-resolution volume data and non-linear transfer functions with high frequencies, without the performance overhead” (see his publication and examples on the link). Also there were several methods grown from his approach applying to particular cases (for lighting, shadowing, to large 3D datasets etc.), but it is still inefficiency for 4D (moving) datasets (lookup tables should be recalculated each time).
I still interesting with yours technical approach to render the volume in Unity. I just realized volume rendering through the 2D textures in WebGL. I want to adapt one in Unity and realize above-mentioned improvements. If you are interesting with collaboration you have my email.
Regards,
Ruslan
July 14th, 2011 at 4:06 pm
It’s worth noting that pre-integrated texture lookups helps both the sliceplane and raymarching methods.
January 31st, 2012 at 3:14 pm
How is this coming along? Excited to see your current status and implement it. Great Work– the best work I have seen– doing things that *I am interested in seeing!