WebAssembly (Wasm) is the not-so new standard to run high performance code in the browser, in the client side. You can think of it as an optimized Virtual Machine (VM), which translates intermediate code, bytecode, into the target machine architecture, all of it within a secured sandbox. It is not a replacement for JavaScript, it is not the silver bullet, but has allowed us to think of the Web as another desktop target for .NET, in the same way Windows, macOS, Android or iOS, among other ones, already are.
You can right now run your .NET Core Console app embedded in a HTML file, make web requests, access the File System (FS), draw stuff with WebGL (the Web branch for OpenGL), and a lot of more things, just because it is the .NET runtime being executed in Wasm. There are still steps to be taken in order to make things smoother for us the Developers but, at the same time, begins to be mature enough for using along projects.
TL;DR: We are announcing our experimental glTF on-line viewer made with Wave Engine 3.0, powered by WebAssembly. Try the demo! http://gltf.waveengine.net
During dotNet 2019, on past june, we presented our initial support for WebAssembly (Wasm), showcasing our WebGL.NET library which serves us to draw at a low-level layer. On the following months we worked on refactoring the OpenGL piece into a platform-agnostic WaveEngine.OpenGL.Common one, from where WaveEngine.WebGL was born. By the end of the year our visual tests started to pass, and we were ready for testing such in a more real scenario.
Our current WebGL backend relies on its 2.0 version, which is supported by most of the browsers: the new Edge (based in Chromium), Firefox and Chrome. Although Safari allows to enable WebGL 2.0 through its Experimental Features menu, it is not 100% completed and breaks when running our app. If you are on macOS, please try Chrome or Firefox; on iOS, it is currently not possible because every browser relies on WebKit, Safari’s foundation.
The on-boarding experience
glTF viewer is a SPA (Single Page Application, a website run on a single page) which works entirely in client side, powered by Mono’s support for Wasm, which will be included in .NET 5. glTF is the nowadays standard for 3D models, and can be viewed on-line by simply drag & dropping such inside —we’ve included a demo-mode for those without a handy file close. Its main features pack:
support for glTF 2.0 (*) in different flavours: plain glTF, glTF-Binary (.glb) and glTF-Embedded
(*) it may happen models fail loading: we are working on making the import process stronger, and would help us if you report us any issue may find (thanks in advance!)
load .glb files from external links: you can show models to others by just sharing a single link (example)
manipulate the model with mouse or fingers, thinking on mobile devices for this last
This article will visit a few caveats we found during the development, and how we surpassed them finally. We hope you enjoy reading such and, hopefully, will learn something new in between.
There is no File System (FS) in the Web
That is not actually true. However, that is the point where we found our-selves when began to work on this project: Wave Engine relies on the concept of content, a path in the FS where the assets are placed. Such assets are, in most of the cases, preprocessed in compile-time. When an app starts the assets are ready to be consumed by the engine.
Thinking on dropping a set of glTF files within a web page, we needed to drive through the content pipeline in order to process the model through our glTF importer. Along with Kenneth Pouncey (thank you!), from Mono, we had already rewrote in C# the Emscripten’s tool to build a Virtual FS (VFS), Mono.WebAssembly.FilePackager, by specifying a local path; however, how do we write there files coming from the outside? Emscripten has solved this already, by offering a FS API in JavaScript, in a flavour similar to common I/O operations:
function writeToFS(filename, typedArray) {
// TODO would there be any other way without handling exceptions?
try {
FS.stat(DropAbsoluteDirectory);
} catch (exception) {
FS.mkdir(DropAbsoluteDirectory);
}
let destinationFullPath = DropAbsoluteDirectory + '/' + filename;
let stream = FS.open(destinationFullPath, 'w+');
FS.write(stream, typedArray, 0, typedArray.byteLength, 0);
FS.close(stream);
}
This functions writes the bytes at typedArray into Emscripten’s VFS
Solved this, the next issue we found was how to overcome the glTF files dropped were not processed in any way, and Wave Engine “does not support” reading such on the fly. The quotes are intended, as we do support such: dropping such in the Editor renders the model immediately, but there is some magic underneath.
We started exploring to consume WaveEngine.Assets namespace inside and app, instead of just from the Editor, which was its natural environment. And voilá, it worked! When a .glb file (the single binary format for glTF) is dropped:
it is imported by “decompressing” its content (textures, materials, etc.) into the FS, and
it is exported by generating .we* files ready to be read by Wave Engine
One of those files is the actual model, which later is used to instantiate the entire entity hierarchy added to the scene. It is not a heavy process when run on the desktop but, when used from Wasm, it takes some precious seconds. Mono has recently added support for multi-threading but, until such will not be broadly adopted by most common browsers, we still cannot rely on it, although will definitely help us reduce such time, as we currently process items one by one.
One of the most time and memory consuming tasks above process takes is texture importing, which will cover next.
Getting image pixels
Of the large bunch of models we have tested those days, the most common textures are made of 2048 x 2048 pixels. When we read such the pixel format is expressed as RGBA, which means 4 bytes per pixel. Thus, if we want to allocate space to read the image in memory we need arrays of 2048 * 2048 * 4 bytes, which is a lot. We have found in some minor cases 8K textures, which make such even worse.
Allocating memory it-self is not a problem, Wave internally depends on ArrayPool for importing textures, which at least makes that smoother. Textures are handled by our ImageSharpImporter, SixLabors’ ImageSharp, which we chose mainly because of its cross-platform feature but, under Wasm, there is a large room open for improvement. With big numbers, decoding a 2048 pixels side image can take more than 10 s, which breaks the experience with no doubt. (We have found also blockers with AOT, but eventually workarounded such by disabling the stripper on their assemblies.)
How, then, can we read images faster? For our WebGL.NET samples gallery, our friend Juan Antonio Cano consumed Skia through some initial .NET bindings, and already solved such by taking some hundred ms. However, the current state of such bindings, made by Uno team, were not compatible with vanilla Mono Wasm, thus we looked for an alternative thinking on maintenance in a future. Also important, we only needed the small piece to decode an image, and nothing else.
It turns out CanvasKit (“Skia in Wasm”, quickly), exposed such piece, and has a JavaScript interface. We made some tests in the CanvasKit playground and looked promising. Then, our CanvasKitImporter was born —replacing ImageSharp one.
private JSObject DecodeImage(Stream stream)
{
if (!stream.CanSeek)
{
throw new ArgumentException("The stream cannot be seeked", nameof(stream));
}
stream.Seek(0, SeekOrigin.Begin);
JSObject image;
using (var memoryStream = new MemoryStream())
{
stream.CopyTo(memoryStream);
using (var array = Uint8Array.From(memoryStream.GetBuffer()))
{
image = (JSObject)canvasKit.Invoke("MakeImageFromEncoded", array);
}
}
return image;
}
CanvasKitImporter.DecodeImage() passes the underlying byte array to CanvasKit, which decodes the image
Loading models like FlightHelmet took from minutes (15-30) with the tab frozen to less than 20 s. And it still takes too much for us, but we must recall the asset export & import process is untouched from the desktop code. We initially though the ArrayPool.Rent(length) call was forcing Garbage Collector (GC) to pass and incurring in some seconds but, after isolating such calls, it is not the culprit at all. We still need to investigate here more in depth.
FlightHelmet model loaded (notice the ilumination)
To the Web and beyond
Not everything is solved: loading time for very big models must be reduced, memory allocation must be decreased too, our WebGL abstraction can be faster as well. Nonetheless, this glTF viewer is our first public project made with Wave Engine 3.0 for the Web.
We have pursued such during some time but the scenario was still not ready for the jump. Nowadays, we see a bunch of possibilities for helping our customers to take visual experiences into the browser, adding Web to the list of officially supported platforms.
If you think we can help you reach the Web too, we are here to listen. Oh, and if you found any issue, please report it. Thank you for reading.
UPDATE (April 19th): The document now reflects new changes introduced in Wave Engine v. 2.1.0.
Introduction
Default Wave Engine launcher project for iOS assumes that your game will run as a stand-alone view inside your application. However, what if you want to mix native views or navigate between different controllers? Is there a way to join the native world with your scene’s one? The answer is yes. Keep reading this article, and you will learn how to accomplish it.
Development
Create a Project at Wave Visual Editor
First step is to create a new Wave Engine project using Wave Visual Editor. You can follow My first application which will guide you step by step. Please, remember to include iOS as target platform when creating it (you can later create it as explained here).
Open the iOS launcher solution (you’ll find the Solution file inside the project’s root folder), we’ll write some extra code to achieve our goal.
Populate the Storyboard
You’ll notice there’s an existing Storyboard inside Controllers folder. From this point of view, the app behaves as any other iOS one, containing view controllers within such Storyboard.
If you open Main.storyboard, there will be the Main Controller, with a GL View in the end, where Wave Engine is actually rendered. On top of that one, there is another “empty” view (well, it just contains the GL one). This one is where you can drag and drop native controls.
So far so good. Wave Engine is, starting from v. 2.1.0 (Hammer Shark; complete list of versions), already integrated into a common iOS project. Please keep reading to discover a small sample, and how the communication can happen between both worlds.
Sample
The principal objective of our sample solution will be to communicate some native side views with our scene, allowing them to change our game’s entities behavior.
Following with above steps, still in the Storyboard, we’ve also added an UISwitch for pausing the game scene, in order to highlight how the communication between native iOS and Wave Engine can be done.
Our scene is quite simple: a fixed camera pointing to a cube, which’s endlessly spinning. Such can be entirely set-up within Wave Visual Editor (remember My first application; Load a primitive can be helpful as well). Due to the nature of the demo, pausing the scene is achieved simply by deactivating the Spinner component, child of the cube Entity. This can be done through its IsActive property.
The communication happening from the iOS UIViewController to the Wave Engine Scene happens in the following chain:
User touches the UISwitch which fires the AutoRotateSwitchChanged() “event” (actually it’s not an event, but a partial method automatically created from the Storyboard Designer at Visual Studio/Xamarin Studio)
Such contains a reference the Game it-self, so we’ve added a method to it: game.UpdateAutoRotation()
The Game knows about the Scene’s being in play, so can call custom code from those, which we’ve done through: scene.UpdateAutoRotation()
Apart from the integrations currently available, we’ve seen how native iOS controls can coexist with Wave Engine, opening a world of opportunities, such like CAD apps. The entire Source code can be downloaded from GitHub:
