August 27th, 2023, 21:03 UTC

allenap.me Rust JavaScript Parcel

Experimenting with Parcel for allenap.me

tl;dr Parcel does some impressive stuff but does not work well outside of a JS-centric web stack, and it can take a significant effort before you know if it’s going to work for you or not. For me, it did not.

One day I was thinking…

I’m pulling CSS and JS into the server that runs this site like so:

fn render() -> maud::Markup {
    maud::html!(
        (maud::DOCTYPE)
        html {
            head {
                title { "allenap.me" }
                script { (include_str!("code.js")) }
                style { (include_str!("styles.css")) }
                // ...
            }
            body {
                // ...
            }
        }
    )
}

(This uses maud, a library for embedding HTML into Rust that I like very much :chefs-kiss:)

The example above is not exactly how I do it, but it’s illustrates my approach closely enough. The point is that I’m including snippets of JS and CSS into HTML, and I generate that HTML in my code. It’s not coming from a template or a JS framework, and it’s not static.

Anyway, so this is the thinking bit: those include_str!(...) macros – a more capable macro could do stuff, like process SCSS into CSS, or minify JavaScript, or compile TypeScript or Elm. I’m not the first person to have this idea: see turf 🌱 for the SCSS→CSS one, for example.

Now turf 🌱 (with the nice emoji) uses lightningcss which is part of Parcel. Ah! Parcel looks cool. Yes, it has a nice website – I need to learn from that – but it’s cool for more than just a dashing front page.

Parcel

If you don’t know, Parcel builds a lot of things – like SCSS, TypeScript, Elm – and can minify too, tree shake (old news for Elm; been doing that for ages 🥱), etc. You can give it, for example, an HTML file, and it’ll find the JS, TS, Elm, CSS, images, etc. referenced therein and do its magic. It’ll write out those new assets, including your HTML, into another directory, all wired up perfectly. It’s cool 😎

I just need to plug this into my code: maybe I’ll need a new macro or something to trigger Parcel. First question: how do I invoke it?

$ parcel --help
Usage: parcel [options] [command]

Options:
  -V, --version               output the version number
  -h, --help                  display help for command

Commands:
  serve [options] [input...]  starts a development server
  watch [options] [input...]  starts the bundler in watch mode
  build [options] [input...]  bundles for production
  help [command]              display help information for a command

  Run `parcel help <command>` for more information on specific commands

parcel build is not what I need

The build command is only for release/production builds, but I definitely want debug builds. That’s not going to fly.

parcel watch is also not really what I’m looking for (but it’s going to be what I end up with)

watch will compile stuff I want, but then it’ll linger afterwards. If I want to invoke Parcel from a macro or during a Rust build that’s not going to fly either. But… there is an option that might work:

  --watch-for-stdin          exit when stdin closes

Maybe I can get it to run once?

$ parcel watch --watch-for-stdin foo.js < /dev/null
STDIN closed, ending

$ echo hello | parcel watch --watch-for-stdin foo.js
STDIN closed, ending

$ printf 'hello\ndo not go\n' | parcel watch --watch-for-stdin foo.js
STDIN closed, ending

Ugh, nope. Okay, that’s frustrating, but a seed has been planted – which is going to lead me to change my approach. I will come back to this.

parcel serve is definitely not what I need

Parcel has one more command: serve. Can I use this? Quick answer: no. This is like watch but it also starts a web server. One thing I already have is a web server, and it’s kind of the point that everything goes through my custom web server – it’s where a lot of the functionality is.

Not giving up on Parcel

I am a bit disappointed. Parcel’s focus seems to be towards a JavaScript-centric web stack, and it’s not easily composable into other stacks – even though it could be. For me, for example, all I need is a build command that produces a debug build.

Parcel has a lot going for it so I’m not going to let go just yet, but the honeymoon is definitely over.

Aside on that “JavaScript-centric” thing

It doesn’t take Sherlock to figure this out. The website URL – https://parceljs.org – more than hints at it. The core of Parcel is supposedly Rust, but a lot of it is actually JavaScript… digging around the Parcel repo and I’m not sure even the core is Rust… it looks like a JS app that has some Rust parts 🧐 Some components are published to crates.io, like lightningcss, but I had the preconception that it was an all-or-mostly Rust project. To be fair and clear, the Parcel website doesn’t overpromise on this:

Parcel’s JavaScript compiler, CSS transformer, and source maps implementation are written in Rust for maximum performance.

I am not a fan of the JS/Node.js ecosystem. It has burned me too many times and continues to do so. I would have long ago kicked JavaScript out of my toolbox and forgotten about it, except that it’s ubiquitous and necessary for almost any kind of web work. Even paper cuts become gaping wounds with repeated exposure.

JavaScript is also a language that could have been way, way better after 28 years, especially with the number of eyes on it and the resources that have been thrown at it. For the tip of the iceberg see: Set and Map for examples of half-finished but standard APIs; undefined and null for the billion dollar mistake squared; and equality comparisons and sameness as formalising of past mistakes. JavaScript long ago took the path towards becoming a grandfathering swamp.

Parcel is still pretty cool; I’m not here to bash it. There’s a ton of utility in there, and it shields me from so much of the JS ecosystem nonsense in an elegant way. I probably ask for too much, but I had been hopeful that Parcel was also part of a trend to move web tooling away from the JS hegemony. Maybe it is, but it’s a tiptoe not a stride.

Rethinking

Maybe I can run parcel watch anyway, outside of my Rust build, and include_str!(...) the processed CSS, JS, etc. Part of my development process is to use cargo watch to automatically rebuild and restart my web server after a change. This would keep working: it would see updates coming from Parcel and trigger a build+restart.

What are the downsides?

I don’t know enough at this point to say with certainty, but a few things come to mind:

  • I’m not sure that Hot Module Replacement will work, but I can live with that. I might be able to get that working another day. Update: looks like this works anyway. Nice one, Parcel.

  • It’s another process to run when developing. That’s also something I can live with. Indeed, a tool like Hivemind or Overmind makes that a non-issue.

  • There’s no way to sequence a debug build to happen before the Rust build, so the latter could fail if compiled assets are not yet in their expected locations. Both Parcel and Cargo will be watching for filesystem changes so I think this kind of error should resolve itself fairly quickly, but there may be edge cases.

Those sound surmountable; no reason to stop now. But, as I will discover, I will have to rethink and rework a whole lot more – and it still will not work.

Turning my app inside out, or: How ES Modules are a pain in the backside

I soon discovered that <script>someJavaScriptFromParcel</script> is semi-useless.

I want to put my page-specific code in the page, inline. Also in the page is a fragment of code to initialise, e.g. something like makeToolbars("header > p.tool-bar") (I want that selector to live apart from the rest of my code, close to the element to which it refers, so that if I change one I at least have a reasonable shot at remembering to change the other). Hence, what I was trying was: code up a view.js (say) on the filesystem, get Parcel to process it, then use include_str!("dist/view.js") to put it into my page (to reiterate: this is in my Rust handler code, where the page is rendered dynamically).

However, by The Power of ESM, that code loads, runs, whatever, but cannot be referred to by another <script> in the same page. Instead, it seems I have to refer to my page-specific code in a separate file that is then loaded separately, meaning I need a separate route on my web server to serve that code separately. Instead of a compile-time guarantee that my code is where I need it, I need some indirection to really get the benefits of a modern JS stack.

Read MDN’s page on JavaScript modules and try not to despair.

Now I need to do more work. ES Modules are apparently what I need. Maybe so, but they suck in this moment. I could put my selector into the library code, or I can put my code – that is specific to this single page – on a separate route and load it separately, etc. I’m being pushed towards the latter.

Aside: I had to walk away from this project for a day after discovering the above (and having wasting hours). Writing this is therapy. The whole JavaScript web stack is so corrosive to one’s health, but we keep building upwards and outwards, metastasising.

Trying to put this all together any way I can

I add a route to serve files from dist, i.e. where Parcel puts its output. My test module:

export function makeToolbars(selector) { /* ... */ }

I can import this into the page:

<script type="module" src="/dist/src/posts/view.js"></script>
<script type="module">
  import * as foo from '/dist/src/posts/view.js';
  console.dir(foo);
</script>

But disconcertingly, that console.dir does not print out any exports. Indeed, trying to import:

<script type="module">
  import { makeToolbars } from '/dist/src/posts/view.js';
</script>

yields only an error in the browser console:

SyntaxError: The requested module '/dist/src/posts/view.js'
  does not provide an export named 'makeToolbars'

(Aside: seriously JavaScript, SyntaxError?! No. A syntax error is when I type something like const foo = return 7;. A missing export from a module fetched from elsewhere – this could have been a third-party’s module over which I have no control – is not an error of syntax.)

I’m guessing that makeToolbars has been tree-shaken or something, i.e. Parcel is being overzealous and I need to dial it down a bit.

It seems – a guess – that I need to read about targets. This document is long and confusing and a mess of overlapping concepts, as well as being JavaScript-centric. Parcel can process any number of different things but most of the configuration seems to be about the myriad ways in which it should process and produce JavaScript.

I think Parcel wants me to put configuration in package.json. Ugh, please not package.json, but okay, I want to get this working. I add:

  "targets": {
    "default": {
      "distDir": "server/dist",
      "source": [
        "./server/assets/**/*.js",
        "./server/assets/**/*.css",
        "./server/assets/**/*.svg",
        "./server/src/**/*.js",
        "./server/src/**/*.css"
      ],
      "context": "browser",
      "isLibrary": true
    }
  }

That "isLibrary": true bit is what I’ve got my hopes pinned on:

When set to true, the target is treated as a library that would be published to npm and consumed by another tool …

i.e. unused imports – at least from Parcel’s purview – won’t get shaken off the tree.

Let’s give this a go:

$ parcel watch
🚨 Build failed.

Error: ENOENT: no such file or directory, stat '/Users/gavin/GitHub/allenap.me/something-else/run/bin/python'

Um, that whole directory tree has got nothing to do with this. But I did do something different this time: I invoked parcel watch from …/allenap.me instead of …/allenap.me/server as I had before. Let’s try what worked before:

$ (cd server && parcel watch)
🚨 Build failed.

unknown: Could not find entry: /Users/gavin/GitHub/allenap.me/server

Ugh.

Conclusions

That last error broke me. It’s enough: I’m ending this experiment. I’m not learning anything new at this point – which is 90% of the point of this – and I’m losing time when I could be doing something useful. I’m getting bitter but I don’t want to drag down Parcel. I get the feeling that if it fits your needs and your setup it’ll be a tremendous boon. But for me, writing a custom server in Rust, it’s not helpful, and sadly it took a couple of days of plugging away to figure that out. I may be able to make use of some of Parcel’s components individually, like the CSS processor, but for now I’m going to do without. My JS and CSS will be fine unminified and without having their tree shaken.