CLI only shows assets emitted for initial build, not subsequent watcher-trigger builds

[markfinger]

From memory, the compiler wrapper gets returned by the build function. So something like

var wrapper = build({
  // ...
}, func...);

wrapper.onDone(function(data) {
  console.log(
    // file paths, or somesuch
  );
});

[donasaur]

Hey @markfinger , I'm not sure if this is related to the issue that you're describing, but it looks like the callback passed into build is only invoked once even when watching for changes.

Is there a way to have the callback invoked multiple times?
Or maybe emit a 'build' event that we can listen to to get access to the data object provided to a callback.

[markfinger]

Yes, in principle, but it becomes a bit problematic once things are run in a worker process.

For context, most of the system was built to work on a request/response cycle, eg: refreshing a browser and requesting the current state of the build. The CLI was a bit a hack job that I needed on a particular project.

Hmm, there are lower layers that expose that sort of an interface, so we'd need to either expose them higher up, or just poke around the currently running Watcher and Wrapper instances at runtime and hook in to them.

[markfinger]

If things are running in a worker process, we'd need to expand the amount of messages that the workers send back to the parent.

[markfinger]

Hmm, a hack to get around the architectural limitations would be polling via a setTimeout that begins after every build request. Internally, webpack-build caches the stats and error objects, so there wouldn't be too much overhead.

[donasaur]

Ah ok. Is the semantic of a request a single invocation of the compiler (i.e., compiler.run(..)), where compiler is a webpack instance with a particular options object? I'm also assuming that at most one worker deals with a single request.

In that case, would it be possible to do the following:

1. have a worker process send a message (eg, print to stdout) via a console.log inside a callback passed into compiler.run(). maybe the worker includes within the message the entire data object that webpack sends back on a finished compilation
2. have the parent process listen to that particular message and then invoke the original callback the API user provided with the data object

[markfinger]

Requests should be considered as an invocation of webpack-build. Incoming requests are matched to compilers by serialising the options object, hashing it, and then using that hash as a key.

When a request is passed in to webpack-build it looks to see if there is a compiler already running that matches the request and passes the request down to it. If there is no matching compiler, it spawns a new one and then passes the request to it.

If worker processes are available, it looks to see if there is a worker that has previously handled the request, then sends it to that process. If no worker has handled it, it picks the next available one and requests it to start handling it. Jobs are assigned to workers in sequential fashion, so it's possible for workers to handle multiple compilers.

Internally references are maintained to the compilers and little state machines (the Wrapper instances) are wrapped around each compiler. The wrappers track the compiler's actions, mostly to detect when to cache the output (to memory and disk) and when to invalidate (eg: when a watcher detects a change). When a request comes in, it serves up any valid cached data or stores the callback and waits for the compiler to reach the done stage, at which point it caches the data and serves it up to the callback.

---

The workers have a messaging interface built in, so it should be easy enough to add another message type, if needed.

This might be the best point to intercept and pass a message up: https://github.com/markfinger/webpack-build/blob/master/src/wrappers/Wrapper.js#L218-L228

Here's an example of conditionally sending messages if the code is running in a worker: https://github.com/markfinger/webpack-build/blob/master/src/hmr/index.js#L22-L27

This is where the parent process receives messages: https://github.com/markfinger/webpack-build/blob/master/src/workers/Worker.js#L112-L147

Also, not sure if it's of use, workers share the IO of the parent process, so you can console.log and it'll print in your terminal.

If you invoke your process with DEBUG=webpack-build:* node ..., you'll get a pretty massive amount of logging, but it might help to follow things.

[imathews]

Something along these lines would be incredibly useful to me — basically a way to listen for a buildStart and buildFinish event, allowing me to tie into express middleware and not serve assets until the build is completed (when watching files, similar to webpack-dev-middleware, but without the slow server startup times).

Maybe this is doable already, though haven't been able to figure it out if so...

[markfinger]

@imathews you should be able to just invoke the wrapper and it'll execute your callback when the build's done.

// ...
var webpackBuild = require('webpack-build');
// ...

// assuming you wanted it in a middleware
app.use(function(req, res, next) {
  webpackBuild({
    // ...
  }, function(err, data) {
    if (err) return res.status(500).end('Build error: ' + err.stack);

    // Attach the data to the response
    res.locals.build = data;

    // Assuming you define `staticUrl` and `staticRoot`, the `urls` prop should contain
    // urls grouped by entrys
    console.log(data.urls); // urls

    next();
  });
});

This package was designed to handle that exact scenario - unfortunately, to the detriment of other use cases :(

[imathews]

@markfinger Thanks!! This is exactly what I needed, working perfectly.

For what it's worth, might be good to throw an example to this effect in the README... definitely wasn't obvious to me. (I was calling the build function once, rather than every time a request was made).

Cheers.