-
Notifications
You must be signed in to change notification settings - Fork 1
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add support for JIT builds #27
Comments
I think we do need to enable passing flags to a JIT compiler somehow, and this may come in handy since we find ourselves repeating our options a lot. Here's a simple example from a fairly typical package: {
"build": "ez-build --production --flags es2017,add-module-exports",
"build:dev": "ez-build --interactive --flags es2017,add-module-exports"
} Babel and the likes "solve" this by magically picking up files like gcc hello.c @gcc-opts This would pick up options from the file gcc hello.c $(cat gcc-opts) What I like about this approach is that it's explicit. There are no magical files, and furthermore it's not JSON which is a terrible format for these kinds of things. Furthermore, you can easily have multiple files: gcc hello.c @production @test @whatever And they would all expand in the specified order, not according to some arbitrary lookup algorithm that no one cares to remember. (This has already bitten us.) |
Inspired by this [comment][1] on #27, this change adds the ability for ez-build to read options from files, and placing their contents precisely in the order the user intended. This is essentially the same as just printing the contents of the files verbatim, taking care to allow newlines by splitting on *all* whitespace. This allows us to re-use ez-build configuration across scripts, in a very predictable and explicit manner, as opposed to having magical lookup rules that can be hard to reason about. [1]: #27 (comment)
So #37 gives us a way to specify build options in a file, but it doesn't really help with these scenarios, where really we just want to load a module and have it hook into Node's module loading process. Not sure what to do about that. One thing that could be done I suppose is add support for a special ez-build environment variable, which would just be a list of options or point to an options file. This could let us do something like this: EZ_BUILD_OPTS="@build.opts" tape -r ez-build/JIT test/*.js But it doesn't really help with the first scenario, unless you also run that program with the environment variable set at first. Another option perhaps, would be to enable special loaders once the JIT builder has been registered – essentially looking for patterns in the module path. It could look like this: tape -r ez-build/JIT -r @build.opts test/*.js Following on from the precedence set in #37, we'd stick with the
I kind of like this, and would neatly solve both scenarios described in the original post. It may be presumptuous to think we can just hijack the |
@mstade Is there anything we can do in the community to help out or get involved with JIT builds in ez-build? 💭 It would be great to get JIT builds in place for me, as it would help me clean up complexity in managing |
@matthewdunsdon sure, a PR is always welcome! There are at least two issues two solve:
Another option that I used with some success is to actually just set up an interactive build, where tests are also built and placed into either |
In some scenarios it's not desirable to build an entire project, or build files on change, but to build separate files on the fly as they are being loaded. This is a very common scenario in testing, where the test suite may need to be built, and the targets under test may also very well need to be built. Many test runners include the ability to load a module before running any tests, that enable such functionality – tape for instance, has this very feature.
Expected Behavior
When running tests, I expect to be able to load a special module, that hooks into the runtime environment and changes how modules are loaded. The precedent set by babel-register is good, and easy to work with.
There are essentially two scenarios that need to be covered:
Registered as the first
import
(orrequire
) and thus overriding any subsequent imports – e.g.:Preloaded by node or another runtime, e.g.:
node -r ez-build/JIT program.js
ortape -r ez-build/JIT test/*.js
Passing options to ez-build is probably overkill, but could be implemented after the fact anyway.
For implementation inspiration, the babel-register source is a fairly simple read.
Current Behavior
Currently, it's not possible to run ez-build this way. You can run it interactively, which will recompile specific files as they change. Essentially this is the same as the JIT build explained above, it's just a different way of getting the names and paths of files being loaded.
Not having this functionality makes it more difficult to accurately test projects built with ez-build, since you effectively have to recreate the internal configuration of ez-build which is both difficult to get right, and may very well change over time which makes it difficult to maintain.
Considerations
Ideally, tests should test the optimized output of ez-build, not just intermediate output – however this would add considerable overhead in a development environment where you're constantly re-running tests. The overhead may not be very much, perhaps a few seconds at worst, which isn't too bad in a CI environment but is very cumbersome in a development environment. It's not immediately apparent to me how to achieve this. One way might be to have two different JIT modules, one for development, which really just JITs files as they are loaded, and a "production" JIT module which would first perform a
--production
build of the project, then remap all modules to the optimized output as opposed to individual files, and finally JIT any other modules being loaded (i.e. test modules.)Ideally though, there should be no functional differences between intermediate builds and optimized builds – and if that's the case, this whole point is moot since it's fine to just JIT everything and never test the optimized output. That's clearly the cowboy option.
The text was updated successfully, but these errors were encountered: