How WebSharper.Warp works behind the scene

How WebSharper.Warp works behind the scene

Lately I’ve been very happy about how WebSharper.Warp allows me to iterate quickly and without pain. Last week, I covered how we could use WebSharper.Warp to build prototypes quickly. Check it out if you haven’t read it yet. Today, I decided to explore how WebSharper.Warp actually works behind the scene.
By looking at how WebSharper.Warp works, we will learn two things:
  1. The process of compiling F# to WebSharper using WebSharper.Compiler
  2. When does the JS files get created

Exploring WebSharper.Warp

WebSharper.Warp is a library which allows us to boot a sitelet from a .fsx file and run the sitelet from the FSI.
Here’s a short example - if you want better explanation, I covered it in last week post.
The following script can be run in a .fsx. It boots up a SPA served on localhost:9000, with JS code and makes one call to a backend endpoint to get a Hello!. We basically get all the power of WebSharper to be run from FSI. It makes it easy to rapidly scribble some prototype and run a complete WebSharper webapp.
#I "../packages/"
#load "WebSharper.Warp/tools/reference-nover.fsx"
open WebSharper
open WebSharper.JavaScript
open WebSharper.Sitelets
open WebSharper.UI.Next
open WebSharper.UI.Next.Html
open WebSharper.UI.Next.Client

module Remoting =
    [<Rpc>]
    let sayHello() = 
        async.Return "Hello!"

[<JavaScript>]
module Client =
    let main() =
        View.Const ()
        |> View.MapAsync Remoting.sayHello
        |> View.Map text
        |> Doc.EmbedView

module Server =
    let site =
        Application.SinglePage (fun _-> 
            Content.Page [ client <@ Client.main() @> ])


do Warp.RunAndWaitForInput Server.site |> ignore
How does it work?
WebSharper.Warp is quite fascinating. All the code is contained in a single file Warp.fs. It combines three steps:
  1. Compiles the files to JS,
  2. Boots up a server
  3. Serves a single endpoint.
It also provides some helper functions to rapidly create sitelets. It is interesting to look at how WebSharper.Warp works as it is almost the same code that runs during MSbuild when unpacking scripts and content files.

Using WebSharper.Compiler

The main function in WebSharper.Warp is the compile function. It is located in the Compilation module and uses WebSharper.Compiler.
let compile (asm: System.Reflection.Assembly) =
    let loader = getLoader()
    let refs = getRefs loader
    let opts = { FE.Options.Default with References = refs }
    let compiler = FE.Prepare opts (eprintfn "%O")
    compiler.Compile(asm)
    |> Option.map (fun asm ->
        {
            ReadableJavaScript = asm.ReadableJavaScript
            CompressedJavaScript = asm.CompressedJavaScript
            Info = asm.Info
            References = refs
        }
    )
This function is used to compile a dynamic assembly which is exactly our case since we are handling a running in FSI. The output of compile is a CompiledAssembly which exposes intesting members like ReadableJavaScript, CompressedJavaScript and Info.
type CompiledAssembly =
{
    ReadableJavaScript : string
    CompressedJavaScript : string
    Info : WebSharper.Core.Metadata.Info
    References : list<WebSharper.Compiler.Assembly>
}
The first part of the code in the compile function is to get the references from the current assembly with getRefs.
let loader = getLoader()
let refs = getRefs loader
It does a bunch of recursive calls to get the full tree of references (references of references etc) by doing some clever filtering to avoid duplicated references. The full code of getRefs can be found here. Then it passes those references to a loader which will transform it to a WebSharper.Compiler.Assembly (to not be confused by the CompiledAssembly). These references are then used to build the options needed to instantiate the WebSharper.Compiler.
let opts = { FE.Options.Default with References = refs }
let compiler = FE.Prepare opts (eprintfn "%O")
compiler.Compile(asm)
The compiler is then used to compile and map the result to a CompiledAssembly.
compiler.Compile(asm)
|> Option.map (fun asm ->
    {
        ReadableJavaScript = asm.ReadableJavaScript
        CompressedJavaScript = asm.CompressedJavaScript
        Info = asm.Info
        References = refs
    }
)
Compile is a function from WebSharper.Compiler which can be used to compile quotation code or assemblies https://github.com/intellifactory/websharper/blob/master/src/compiler/WebSharper.Compiler/FrontEnd.fs#L57.
/// Attempts to compile an expression potentially coming from a dynamic assembly.
member Compile : quotation: Quotations.Expr * context: System.Reflection.Assembly * ?name: string -> option<CompiledAssembly>
Here’s a reminder of the full function:
let compile (asm: System.Reflection.Assembly) =
    let loader = getLoader()
    let refs = getRefs loader
    let opts = { FE.Options.Default with References = refs }
    let compiler = FE.Prepare opts (eprintfn "%O")
    compiler.Compile(asm)
    |> Option.map (fun asm ->
        {
            ReadableJavaScript = asm.ReadableJavaScript
            CompressedJavaScript = asm.CompressedJavaScript
            Info = asm.Info
            References = refs
        }
    )
The next step is to write out the ReadableJavaScript and the CompressedJavaScript from the CompiledAssembly. The code which is in charge of that is located under the two functions outputFiles and outputFile.

Output the files

In the previous code, we loaded the references using the Loader. This “loaded assembly” are of type WebSharper.Core.Assembly. They are special in the sense that they carry embedded resources (more on embedded resources can be found in the doc) and also the same properties as the CompiledAssembly, ReadableJavaScript and CompressedJavaScript. The following function extracts all the data contained within a WebSharper assembly:
let outputFiles root (refs: Compiler.Assembly list) =
    let pc = PC.PathUtility.FileSystem(root)
    let writeTextFile path contents =
        Directory.CreateDirectory (Path.GetDirectoryName path) |> ignore
        File.WriteAllText(path, contents)
    let writeBinaryFile path contents =
        Directory.CreateDirectory (Path.GetDirectoryName path) |> ignore
        File.WriteAllBytes(path, contents)
    let emit text path =
        match text with
        | Some text -> writeTextFile path text
        | None -> ()
    let script = PC.ResourceKind.Script
    let content = PC.ResourceKind.Content
    for a in refs do
        let aid = PC.AssemblyId.Create(a.FullName)
        emit a.ReadableJavaScript (pc.JavaScriptPath aid)
        emit a.CompressedJavaScript (pc.MinifiedJavaScriptPath aid)
        let writeText k fn c =
            let p = pc.EmbeddedPath(PC.EmbeddedResource.Create(k, aid, fn))
            writeTextFile p c
        let writeBinary k fn c =
            let p = pc.EmbeddedPath(PC.EmbeddedResource.Create(k, aid, fn))
            writeBinaryFile p c
        for r in a.GetScripts() do
            writeText script r.FileName r.Content
        for r in a.GetContents() do
            writeBinary content r.FileName (r.GetContentData())
For each references, it writes the readable JS and compressed JS into its own file. Then move on to get all the scripts linked from resources, writes those in files. And finally gets all the contents like Css files or images, and writes those in files as well.
For the CompiledAssembly, it is straightforward as the only step needed is to write the readable JS and compressed JS into files.
let outputFile root (asm: CompiledAssembly) =
    let dir = root +/ "Scripts" +/ "WebSharper"
    Directory.CreateDirectory(dir) |> ignore
    File.WriteAllText(dir +/ "WebSharper.EntryPoint.js", asm.ReadableJavaScript)
    File.WriteAllText(dir +/ "WebSharper.EntryPoint.min.js", asm.CompressedJavaScript)
This is why, at the moment, compiling with WebSharper is a two step process:
  1. Compile with msbuild which makes a .dll
  2. Compile that .dll with WebSharper.Compiler which makes a CompiledAssembly

Conclusion

By understanding WebSharper.Warp, we got a better insight on the steps required by WebSharper to compile an assembly. It also showed us how .fsx files could be compiled and translated to JS and at which moment were the JS files actually created. Hope this helped you understand better the mystery behind WebSharper.Warp. Again if you are interested in WebSharper.Warp, check out my last week post on how to quickly prototype with WebSharper.Warp. If you have any comments, leave a message here or hit me on Twitter @Kimserey_Lam. Thanks for reading!

Comments

Post a Comment

Popular posts from this blog

Microsoft Orleans logs warnings and errors

Microsoft Orleans logs warnings and errors Microsoft Orleans is a framework which helps building distributed system by implementing the actor model together with the concept of virtual actors, taking care of availability and concurrency. If you are unfamiliar with Microsoft Orleans, you can look at my previous blog post explaining the benefits of Microsoft Orleans . Even though Orleans promises to abstract the distributed system problems, there are instances where errors arise without us being able to understand what is going on. Lucky us, the logs are well documented… but only for those who can decrypt them. Today I will go through some of the errors and warnings which can be seen from silo and client so that you too can undestand what is going on. Enjoy! The code used to produce those errors can be found on my GitHub https://github.com/Kimserey/orleans-cluster-consul . 1. Client logs Logs on client appears with address {ip}:0 . 1.1. Can’t find implementation of interface An un...
Keep reading

A complete SignalR with ASP Net Core example with WSS, Authentication, Nginx

SignalR with ASP Net Core SignalR is a framework from ASP NET Core allowing us to establish a two way communication between client and server. This two way communication allows the client to send messages to the server but more importantly allows the server to push messages to the client. SignalR makes use of Websocket when available else it falls back to SSE or pulling. Today we will focus on how to setup SignalR to work with WSS, Websocket secure and how we can authenticate the user requesting to connect to our SignalR hub via Webscoket. Getting started with SignalR SSL encryption for Websocket Secure WSS Websocket Authentication with Identity Server 4 SignalR behind Nginx 1. Getting started with SignalR The Hubs are the main components of SignalR. It is an abstraction of a two way communication available for both client and server. Public functions from the hub can be called from the server code and can be called from the client. The frontend NPM package @aspnet/signalr ...
Keep reading

SDK-Style project and project.assets.json

SDK-Style project and project.assets.json Last week I encountered an issue with MSBuild while trying to run it from command line. The issue did not appear when using VisualStudio right click + build but only appeared when using msbuild.exe CLI directly with a clean project. Assets file 'C:\[...]\obj\project.assets.json' not found. Run a NuGet package restore to generate this file. When I first saw the error, few questions came to my mind which I will share today in 3 points: Overview of project.assets.json Slim SDK-Style project Mixing SDK-Style project and old projects Special shoutout to @enricosada who provided me with all the answers regarding the SDK-Style project. 1. Overview of project.assets.json project.assets.json lists all the dependencies of the project. It is created in the /obj folder when using dotnet restore or dotnet build as it implicitly calls restore before build, or msbuid.exe /t:restore with msbuild CLI. To simulate dotnet build (re...
Keep reading