Wave Engine runs on the Web thanks to Mono Wasm

What WebAssembly means for .NET

WebAssembly (Wasm) is the not-so new standard to run high performance code in the browser, in the client side. You can think of it as an optimized Virtual Machine (VM), which translates intermediate code, bytecode, into the target machine architecture, all of it within a secured sandbox. It is not a replacement for JavaScript, it is not the silver bullet, but has allowed us to think of the Web as another desktop target for .NET, in the same way Windows, macOS, Android or iOS, among other ones, already are.

You can right now run your .NET Core Console app embedded in a HTML file, make web requests, access the File System (FS), draw stuff with WebGL (the Web branch for OpenGL), and a lot of more things, just because it is the .NET runtime being executed in Wasm. There are still steps to be taken in order to make things smoother for us the Developers but, at the same time, begins to be mature enough for using along projects.

Continue reading Wave Engine runs on the Web thanks to Mono Wasm

Wave Engine 3.0 Preview 3

We are excited to announce the release of Wave Engine 3.0 third preview!

New features

In this one we have worked hard to include a new template for Web applications. The launch of Mono Wasm has allowed us to deploy your Wave Engine apps to the browser, although there are some limits at the moment (such like native libraries: Bullet Physics, Noesis GUI, etc.) You can read our post about the glTF viewer we have launched to find some details about the implementation.

Also, the engine has been updated to support HoloLens 2, integrating the new APIs and updating the Mixed Reality template.

Furthermore, we have integrated Physically Based Rendering (PBR), enabled by default in the Standard Material, and new photometric lights and cameras that will improve the render quality.

Continue reading Wave Engine 3.0 Preview 3

Wave Engine’s on-line glTF viewer

TL;DR: We are announcing our experimental glTF on-line viewer made with Wave Engine 3.0, powered by WebAssembly. Try the demo! http://gltf.waveengine.net

During dotNet 2019, on past june, we presented our initial support for WebAssembly (Wasm), showcasing our WebGL.NET library which serves us to draw at a low-level layer. On the following months we worked on refactoring the OpenGL piece into a platform-agnostic WaveEngine.OpenGL.Common one, from where WaveEngine.WebGL was born. By the end of the year our visual tests started to pass, and we were ready for testing such in a more real scenario.

Our current WebGL backend relies on its 2.0 version, which is supported by most of the browsers: the new Edge (based in Chromium), Firefox and Chrome. Although Safari allows to enable WebGL 2.0 through its Experimental Features menu, it is not 100% completed and breaks when running our app. If you are on macOS, please try Chrome or Firefox; on iOS, it is currently not possible because every browser relies on WebKit, Safari’s foundation.

The on-boarding experience

glTF viewer is a SPA (Single Page Application, a website run on a single page) which works entirely in client side, powered by Mono’s support for Wasm, which will be included in .NET 5. glTF is the nowadays standard for 3D models, and can be viewed on-line by simply drag & dropping such inside —we’ve included a demo-mode for those without a handy file close. Its main features pack:

  • support for glTF 2.0 (*) in different flavours: plain glTF, glTF-Binary (.glb) and glTF-Embedded
    • here you can find sample models
    • (*) it may happen models fail loading: we are working on making the import process stronger, and would help us if you report us any issue may find (thanks in advance!)
  • load .glb files from external links: you can show models to others by just sharing a single link (example)
  • manipulate the model with mouse or fingers, thinking on mobile devices for this last

This article will visit a few caveats we found during the development, and how we surpassed them finally. We hope you enjoy reading such and, hopefully, will learn something new in between.

There is no File System (FS) in the Web

That is not actually true. However, that is the point where we found our-selves when began to work on this project: Wave Engine relies on the concept of content, a path in the FS where the assets are placed. Such assets are, in most of the cases, preprocessed in compile-time. When an app starts the assets are ready to be consumed by the engine.

Thinking on dropping a set of glTF files within a web page, we needed to drive through the content pipeline in order to process the model through our glTF importer. Along with Kenneth Pouncey (thank you!), from Mono, we had already rewrote in C# the Emscripten’s tool to build a Virtual FS (VFS), Mono.WebAssembly.FilePackager, by specifying a local path; however, how do we write there files coming from the outside? Emscripten has solved this already, by offering a FS API in JavaScript, in a flavour similar to common I/O operations:

function writeToFS(filename, typedArray) {
    // TODO would there be any other way without handling exceptions?
    try {
        FS.stat(DropAbsoluteDirectory);
    } catch (exception) {
        FS.mkdir(DropAbsoluteDirectory);
    }

    let destinationFullPath = DropAbsoluteDirectory + '/' + filename;
    let stream = FS.open(destinationFullPath, 'w+');
    FS.write(stream, typedArray, 0, typedArray.byteLength, 0);
    FS.close(stream);
}

This functions writes the bytes at typedArray into Emscripten’s VFS

Solved this, the next issue we found was how to overcome the glTF files dropped were not processed in any way, and Wave Engine “does not support” reading such on the fly. The quotes are intended, as we do support such: dropping such in the Editor renders the model immediately, but there is some magic underneath.

We started exploring to consume WaveEngine.Assets namespace inside and app, instead of just from the Editor, which was its natural environment. And voilá, it worked! When a .glb file (the single binary format for glTF) is dropped:

  • it is imported by “decompressing” its content (textures, materials, etc.) into the FS, and
  • it is exported by generating .we* files ready to be read by Wave Engine

One of those files is the actual model, which later is used to instantiate the entire entity hierarchy added to the scene. It is not a heavy process when run on the desktop but, when used from Wasm, it takes some precious seconds. Mono has recently added support for multi-threading but, until such will not be broadly adopted by most common browsers, we still cannot rely on it, although will definitely help us reduce such time, as we currently process items one by one.

One of the most time and memory consuming tasks above process takes is texture importing, which will cover next.

Getting image pixels

Of the large bunch of models we have tested those days, the most common textures are made of 2048 x 2048 pixels. When we read such the pixel format is expressed as RGBA, which means 4 bytes per pixel. Thus, if we want to allocate space to read the image in memory we need arrays of 2048 * 2048 * 4 bytes, which is a lot. We have found in some minor cases 8K textures, which make such even worse.

Allocating memory it-self is not a problem, Wave internally depends on ArrayPool for importing textures, which at least makes that smoother. Textures are handled by our ImageSharpImporter, SixLabors’ ImageSharp, which we chose mainly because of its cross-platform feature but, under Wasm, there is a large room open for improvement. With big numbers, decoding a 2048 pixels side image can take more than 10 s, which breaks the experience with no doubt. (We have found also blockers with AOT, but eventually workarounded such by disabling the stripper on their assemblies.)

How, then, can we read images faster? For our WebGL.NET samples gallery, our friend Juan Antonio Cano consumed Skia through some initial .NET bindings, and already solved such by taking some hundred ms. However, the current state of such bindings, made by Uno team, were not compatible with vanilla Mono Wasm, thus we looked for an alternative thinking on maintenance in a future. Also important, we only needed the small piece to decode an image, and nothing else.

It turns out CanvasKit (“Skia in Wasm”, quickly), exposed such piece, and has a JavaScript interface. We made some tests in the CanvasKit playground and looked promising. Then, our CanvasKitImporter was born —replacing ImageSharp one.

private JSObject DecodeImage(Stream stream)
{
    if (!stream.CanSeek)
    {
        throw new ArgumentException("The stream cannot be seeked", nameof(stream));
    }

    stream.Seek(0, SeekOrigin.Begin);

    JSObject image;

    using (var memoryStream = new MemoryStream())
    {
        stream.CopyTo(memoryStream);

        using (var array = Uint8Array.From(memoryStream.GetBuffer()))
        {
            image = (JSObject)canvasKit.Invoke("MakeImageFromEncoded", array);
        }
    }

    return image;
}

CanvasKitImporter.DecodeImage() passes the underlying byte array to CanvasKit, which decodes the image

Loading models like FlightHelmet took from minutes (15-30) with the tab frozen to less than 20 s. And it still takes too much for us, but we must recall the asset export & import process is untouched from the desktop code. We initially though the ArrayPool.Rent(length) call was forcing Garbage Collector (GC) to pass and incurring in some seconds but, after isolating such calls, it is not the culprit at all. We still need to investigate here more in depth.

FlightHelmet model loaded (notice the ilumination)

To the Web and beyond

Not everything is solved: loading time for very big models must be reduced, memory allocation must be decreased too, our WebGL abstraction can be faster as well. Nonetheless, this glTF viewer is our first public project made with Wave Engine 3.0 for the Web.

We have pursued such during some time but the scenario was still not ready for the jump. Nowadays, we see a bunch of possibilities for helping our customers to take visual experiences into the browser, adding Web to the list of officially supported platforms.

If you think we can help you reach the Web too, we are here to listen. Oh, and if you found any issue, please report it. Thank you for reading.

WaveEngine 3.0 Preview 2

New features

We’re excited to announce the release of WaveEngine second preview. In this new preview, the whole engine uses .NET Core 3.0, so all WaveEngine libraries are .Net Standard 2.0 and new project templates are .NET Core 3.0. You can just get .NET Core by simply upgrading to Visual Studio 2019 16.3.

Furthermore, you can now run your WaveEngine projects with Vulkan, DirectX12 or OpenGL/ES graphical backends. In the new WaveEngine launcher, you can choose your favorite backend to work.

Continue reading WaveEngine 3.0 Preview 2

Mixed Reality with Wave Engine

In this tutorial, we will cover the creation of your first Mixed reality app using Wave EngineWe will find out why using this engine is a good choice for creating your holographic experience.

In this app we will load a plane in front of the viewer.

Prerequisites

For simulating your app you will need a Windows 10 PC configured with the following tools:

Project assets

  • Download the files required by the project here.

Continue reading Mixed Reality with Wave Engine

WaveEngine 2.4.1 to 2.5.0

This article is a brief guide to solving the majority of problems that you will find when you upgrade your game project from WaveEngine 2.4.1 version to 2.5.0.

Although WaveEngine has an upgrade tool that runs when you open an old game project with the current WaveEditor 2.5, you can find some issues listed below.

An important point to bear in mind is that Model, ModelRenderer and MaterialsMap components are not longer supported, so you should replace them to use FileMesh, MeshRenderer and MaterialComponent respectively. For more details, you can read the following article. We allowed them to be used in the previous version as deprecated classes.

So, here we go!

Loading Game Info

The most important change is that we need to Load the GameInfo file (.wgame).

WaveEngine 2.4.1

WaveEngine 2.5.0

RenderLayers

Main difference is LayerType properties has changed to LayerId, holding now an int identifier instead of a Type.

WaveEngine 2.4.1

WaveEngine 2.5.0

WaveEngine 2.4.1

WaveEngine 2.5.0

Mainly every property Type LayerType has been changed to int LayerId.

Sampler State

The AddressMode has now evolved to the SamplerState, and can be configured in the Texture Asset, instead of the material or the component.

StandardMaterial

WaveEngine 2.4.1

WaveEngine 2.5.0

WaveEngine 2.4.1

WaveEngine 2.5.0

WaveEngine 2.4.1

WaveEngine 2.5.0

WaveEngine 2.4.1

WaveEngine 2.5.0

WaveEngine 2.4.1

WaveEngine 2.5.0

Dual Material

The DualMaterial class has been removed. It has been merged into the StandardMaterial, which now has Diffuse1 and Diffuse2 properties, among others.

New Model asset workflow

In WaveEngine 2.5.0, the model pipeline has been improved, and as a result, there are a lot of new features that makes easier to work with model assets:

  • glTF support. Now you can use glTF models in addition to FBX, DAE, OBJ, etc.
  • New Animation pipeline. A redesigned Animation3D component with blend animation trees.
  • Skinning & Morphing. Deform meshes using bones or morph targets.

Continue reading New Model asset workflow

Bullet Physics integration in WaveEngine 2.5.0

Bullet Physics is a real-time Physics Engine for VR, games, roabotics, machine learning, etc… http://bulletphysics.org

Now, in WaveEngine 2.5.0, we have integrated Bullet Physics as 3D Physic engine. In previous versions, we use Bepu Physics (a C# physics library http://www.bepuphysics.com)

Continue reading Bullet Physics integration in WaveEngine 2.5.0

Render Layers in Wave Engine Orca 2.5.0

Before this version, when the user wanted to customize some aspect of the rasterizer, blending or the depth states, we needed to create by code a new Layer class, registering it manually and then acceding it by its type. And you needed to implement the SetDevice and RestoreDevice methods, changint the RenderState structure.

Initially we just wanted to integrated them into the Visual Editor, but then we realised that we weren’t allowing the users to use the full potential of the modern graphical cards, so we redesigned it.

So the RenderLayer concept was born, a Visual Editor friendly specification of how the geometry and shaders are presented into the final frame. Our most important goal was to keep the simplicity of the previous implementation but adding the full potential of the modern graphic cards.

Continue reading Render Layers in Wave Engine Orca 2.5.0