Update from Nov 19, 2020: Please note that due to a Google PageSpeed update the guide found here still works, but no longer has the desired effect for Lighthouse. If you are looking for an alternative to improve the performance of your shop, Hyvä Themes might be of interest to you.
Table of Contents
Part One: The “ultimate guide” to Magento 2 JavaScript Bundling
- About Lighthouse Audits
- Benchmarks
- How JavaScript influences the performance
- The Basis — default Magento 2 setup & Patches
- Default Magento Bundling
→ You are here
- Baler
- Magento 2 DevTools browser extension
- Devtools bundling
- integer_net Bundling
- A real-life example
- Conclusions
Baler
Baler is the latest effort from Andrew Levine to write a tool that creates one effective bundle that contains all the JavaScript that is crucial for any page on your site. It then continues to write a dependency graph of all the other JavaScript files and in which cases they are needed. The files from that graph will then be injected into the head of the page to tell the browser it needs to fetch those files as soon as possible, without blocking page-rendering.
One of the things Andrew has put a lot of effort in is to make the tool fast. And it sure is. Creating the main bundle happens through an npm script that's executed through command-line at the root of your project.
As said before, we believe - and truly hope - that this tool will become the standard for Magento 2 frontend optimization. But, in its current state it only creates the main bundle and does not build the dependency graph yet. This means you only get the main bundle, which is at this point just a slight improvement to performance.
Getting started
Base requirements:
- All requirements listed at the 'The Basis - default Magento 2 setup' section except for JavaScript minification
- For Magento versions up until 2.3.4: apply the patches listed before:
- Mixins (up to 2.3.4)
- Fix missing shims and phtml files with mage-ini directives (up to 2.3.3)
Baler main NPM tool
Clone the Baler repository https://github.com/magento/baler/
- Git clone: git@github.com:magento/baler.git
- Download :https://github.com/magento/baler/archive/master.zip
Follow instructions from https://github.com/magento/baler/blob/master/docs/ALPHA.md
- Make sure your node version is >= 10.12.9
- Run
npm run build
from the main Baler directory you just cloned/downloaded - Run
npm link
so that it's installed globally.
Install the Magento Module
Baler needs a Magento module to be installed. https://github.com/magento/m2-bale
Either download/clone the repo and put it in `app/code/` or, better, install via composer.
Using composer:
composer config repositories.magento-baler vcs git@github.com:magento/m2-baler.git
composer require magento/module-baler:dev-master@dev
composer install
bin/magento module:enable Magento_Baler
bin/magento setup:upgrade
Set Magento configuration
Disable all core JavaScript options, and enable Baler Bundling. Again, remove --lock-config if you want it set in database instead of app/etc/config.php
bin/magento config:set dev/js/enable_baler_js_bundling 1 --lock-config
bin/magento config:set dev/js/enable_js_bundling 0 --lock-config
bin/magento config:set dev/js/merge_files 0 --lock-config
bin/magento config:set dev/js/minify_files 0 --lock-config
Compile your themes
I usually run setup:upgrade to remove all compiled assets, and then compile DI and static content:
bin/magento setup:upgrade
bin/magento setup:di:compile
bin/magento setup:static-content:deploy en_US
By the way, did you know there is an excellent feature of Symfony Console, on which the Magento console is based, that allows you to abbreviate those commands?
This one-liner performs all three commands above:
bin/magento set:up && bin/magento set:di:co && bin/magento set:sta:dep en_US
Run Baler
From the root of your project, run: baler build
If you get stuck here, please see the troubleshooting section below.
Run Terser
For best results, minify all JavaScript files Baler did not already minify.
That's it
Done. Go to your frontend and see if everything works. You should see a core-bundle.js file being loaded and a 'requirejs-bundle-config.js' instead of the default 'requirejs-config.js'.
If not, check if the Magento Baler module is properly activated and enabled.
Troubleshooting
Could not find Magento store root
If Baler won't run, because it "Could not find Magento store root from [dir]", it means it failed to find one of the following directories/files:
- app/
- pub/
- lib/
- index.php
In case you run Baler locally with Docker, it means you need to add these directories/files to the list of synced directories. Or run the tool from inside the Docker container.
Errors in your store's frontend
If you see JavaScript errors in your browser's console output, and you are on a Magento version lower than 2.3.4, make sure that you have applied all patches listed before. If problems persist, you might have a faulty RequireJS configuration or you try to run JavaScript code without properly requiring it.
An issue you might run into is jQuery code that is run without properly requiring jQuery first. If anyone put a direct reference to jQuery in a template file or a CMS page/block, you might get errors with bundling that you didn't get before.
For example, this will throw an error if jQuery is not yet defined.
if(jQuery.slickSlider()) {
// initialize a slider
}
Instead, this should be written as:
require([ "jquery", "Magento_Theme/js/slickslider" ], function () {
// initialize a slider
});
Profile
If your frontend runs without issues, start auditing and see what results Baler gives you compared to the default Magento configuration. You should see at least an improvement by a few points.
We saw a greater improvement on actual projects than in the benchmarks we did with default Magento and sample data.
Results
Homepage
Checkout
Implementation
As you'll see on both of the Baler repositories, these tools are currently in alpha state, meaning they are not strictly considered production-ready yet.
We've found they work quite well and heard about some webshops happily running them in production.
The Baler tool itself is incredibly fast, which is a fantastic result from Andrew Levine.
Adding the manual Terser step to it unfortunately is extremely slow. Terser is the best minifier currently out there, but when running it on the highest possible compression setting means you'll be waiting many minutes for each theme/language you run in your store.
You could however play around with the settings (you can look up the settings/flags here) and speed up the process.
We have not used this in our deployment pipeline yet. But you could install Baler locally in the project during deployment. This means you clone the project, npm install
it and execute node_modules/bin/baler
instead.
- Without having tested it - I assume this would work.
Magento 2 DevTools browser extension
Before testing the DevTools Bundling or our 'integer_net Bundling', you need to get the Magento 2 DevTools browser extension.
This extension has a RequireJS Bundle Generator that creates an optimized bundle for your Magento shop by browsing through the shop and keeping track of the used JavaScript files.
The extension is a previous endeavour also from Andrew Levine but unfortunately abandoned in favor of his new tool, Baler (you should by now know what Baler is).
The generated bundle configuration from the toolbar can be used by the RequireJS Optimizer, which is a tool by RequireJS itself, to create (you guessed it) optimized bundles.
Both the DevTools Bundling and our own solution make use of the toolbar. In our own solution, we found some ways to improve the bundles and, most importantly, the way they are loaded.
Installing the M2 DevTools toolbar
You only need to do this once for your device. It will install the toolbar in your browser.
Clone the M2 Devtools repositoryhttps://github.com/magento/m2-devtools
- Git clone: git@github.com:magento/m2-devtools.git
- Download: https://github.com/magento/m2-devtools/archive/master.zip
Follow instructions from Magento 2 DevTools Readme:
- Make sure your node node version is >= 8.x
- Run
npm install
from the main M2-Devtools directory you just cloned/downloaded - Run
npm run build
so that it's installed globally. - Navigate to
chrome://extensions
- Enable
Developer mode
- Click
Load unpacked
- Select the extension folder in the root of this repository
You should now have a new Inspector tab called Magento 2
.
Now open up your Magento 2 shop, using production mode and open the DevTools RequireJS tab.
Following the instructions from M2 DevTools:
- Visit any page in the storefront in a browser with this extension installed
- Open
RequireJS >> Bundle Generator
- Click the
Record
button - Begin navigating to critical pages of your store (Home/Catalog/Product/Cart/Checkout)
- Click the
Record
button again when you are finished - Copy the generated bundle configuration and save it in your Magento 2 website root as
requirejs-config-bundle.js
Good. Now you can proceed either to the DevTools Bundling section or our integer_net Bundling section.
DevTools bundling
You may choose to skip this section if you just want to compare our method with default Magento and Baler. We don't make use of the Magento module used in this section and our method will most certainly get better results than the default implementation described here.
Getting started
Base requirements:
- All requirements listed at the 'The Basis - default Magento 2 setup' section, except for JavaScript minification
- On Magento version up until 2.3.4: apply the patches listed before: Mixins and theShims+mage-init fixes
- The
requirejs-bundle-config.js
file created with the Magento 2 DevTools toolbar, as described in the previous section
Optionally remove Baler:
If you just tested the results with Baler, you might want to clean that up first.
bin/magento module:uninstall Magento_Baler
composer remove magento/module-baler
Install the Magento module
The Magento_BundleConfig module needs to be installed. It's part of the M2-DevTools repository: https://github.com/magento/m2-devtools
Either download/clone the repo and put it in app/code/
or, better, install via composer.
Using composer:
composer config repositories.magento/module-bundle-config vcs git@github.com:magento/m2-devtools.git
composer require magento/module-bundle-config:dev-master@dev
composer install
bin/magento module:enable Magento_BundleConfig
bin/magento setup:upgrade
Set Magento configuration
Disable all core JavaScript options. Remove --lock-config if you want it set in database instead of `app/etc/config.php
bin/magento config:set dev/js/enable_js_bundling 0 --lock-config
bin/magento config:set dev/js/merge_files 0 --lock-config
bin/magento config:set dev/js/minify_files 0 --lock-config
Note that the Magento_BundleConfig actually disables JavaScript minification with a plugin, but I'd consider it good practice to also set it in configuration.
The reason why minification is being disabled is because the RequireJS optimizer will run minification after generating the bundles, and the tool works better if the files haven't been minified already (it might even break)
Install r.js, the RequireJS optimizer
You can either download r.js from the requirejs.org website or, better, install it with npm: npm install -g requirejs
You should now be able to run r.js from command line: r.js
See https://github.com/requirejs/r.js for usage.
Compile your themes
I usually run setup:upgrade to remove all compiled assets, and then compile DI and static content:
bin/magento setup:upgrade
bin/magento setup:di:compile
bin/magento setup:static-content:deploy en_US
By the way, did you know there is an excellent feature of Symfony Console, on which the Magento console is based, that allows you to abbreviate those commands?
This one liner performs all three commands above:
bin/magento set:up && bin/magento set:di:co && bin/magento set:sta:dep en_US
Create the bundles
We can now go to the root of our project where you should have saved your requirejs-bundle-config.js
Now we are going to perform a couple of commands to first copy the generated themes from pub/static/frontend/
to a backup directory and then run the r.js optimizer which will copy the theme back into the original directory with the optimizations and generated bundles.
You should see a long list of output for the files that are being minified and the bundles that are being created.
That's it
To be sure that your new files are being loaded, flush your cache (and maybe even force clean your Varnish).
Now visit your store's frontend and open inspector. You should see `bundles/shared.js` loaded on any page and for any page that has a custom bundle you should see the bundle with the layout handle e.q. bundles/catalog-product-view
Troubleshooting
Bundling fails because certain files cannot be found.
The toolbar will pick up AMD files that might be inside a minified JavaScript file that are not in your RequireJS config. I found it easiest to simply remove those from the bundle configuration and try again.
For example, we found fizzy-ui-utils/utils
in one of our bundle configs, but no such file exists.
Upon inspection, we found define.amd?define("fizzy-ui-utils/utils", ...)
in a library of our project. The RequireJS bundler will try to locate that module and won't be able to find it.
Errors in the frontend
Did you apply the patches?
Is there any JavaScript loaded directly without the needed require([], function(){})
around it?
Some things don't work in the checkout
Mixins might not be working, and the first place you usually find out is in the checkout process.
Again, for Magento versions up till 2.3.4: did you apply those patches?
Other issues
You might have errors in your requirejs-config.js
files, mixins or shims or any other issue. This is unfortunately a case by case issue and you'll have to debug.
Profile
You can now start profiling. Results really depend on your project, but we've seen varying results but for most pages the results are lower than with Baler and even without bundling.
It depends on how JavaScript heavy the pages are really.
The issues we found with this implementation is that the bundles are loaded into the head synchronously and therefore blocking the page-rendering.
What comes next is our implementation where we solve this issue by making the bundles asynchronous.
Results
Homepage
Checkout
Implementation
We've managed to get this into our deployment pipeline installing requirejs locally in the project during deployment. This means you run npm install requirejs
and execute node_modules/bin/r.js
instead. This works well.
The downside to this implementation is that any time you change your shop and JavaScript files are added or removed, you need to create a new bundle config.
In case new files are added or missing from your bundle configuration, this is not a big issue because RequireJS will always fall back to loading a single file if the can't be found in the bundles.
If files are removed, however, you won't be able to create the bundles because r.js
will fail and exit with an error file not found
.
So keeping your configuration up to date is an ongoing process.
You finished reading the third part.
Now would be a good time to skip to Part Four.
How do you wish to proceed?
Part One: The “ultimate guide” to Magento 2 JavaScript Bundling
- About Lighthouse Audits
- Benchmarks
- How JavaScript influences the performance
- The Basis — default Magento 2 setup & Patches
- Default Magento Bundling
→ You are here
- Baler
- Magento 2 DevTools browser extension
- Devtools bundling
- integer_net Bundling
- A real-life example
- Conclusions