How do I ensure assets are present with Rail 7, cssbundling-rails, jsbundling-rails in test mode (RSpec)?

Question

I'm upgrading a large, commercial (proprietary) Rails 6 application to Rails 7. We never used Webpacker, and are instead going directly from bundled gems for things like Bootstrap, to the "Rails 7 way".

It turns out that the "no Node" workflow for Rails 7 has no good answer for components that consist of both a CSS and JS component. In our case, the most obvious offender there is Bootstrap. Faced with maintaining the JS "half" of Bootstrap through import maps and the CSS "half" through something like the old Bootstrap gem or manual vendoring (and yes, there really is no other solution without Node here) we end up back at a full Node workflow.

This is coming together. All front-end components that provide CSS and/or JS were already happily available in NPM, so now that's all managed via package.json & Yarn, with bin/dev driving Sass & esbuild compilation of the SCSS and JS components pulled from either app/assets, app/javascript or node_modules/...; the asset pipeline manifest.js contains only references to the build and images folders inside app/assets as a result.

It feels like a bit of backwards step with all the heavyweight manual maintenance of lists of filenames (wildcard imports no longer supported) along with the complexity of the multiple processes now running under Foreman vs just having things synchronously processed in Sprockets on a per-request basis, but with all that stuff being deprecated/abandonware, it was clearly time to update.

This all works fine in dev & production mode, but what about test? We use RSpec; in CI, there's no built assets and developers don't want to have to remember to run esbuild  or assets:precompile or whatever every time they're about to run rspec. Apart from anything else, it's quite slow.

What's the official, idiomatic Rails 7 solution in a Yarn/Node-based workflow specifically using cssbundling-rails and jsbundling-rails, when you want to run tests with up to date assets?

Answer 1

This is pretty ropey but better than nothing for now; it'll ensure CI always builds assets and also ensure that local development always has up-to-date assets, even if things have been modified when e.g. bin/dev isn't running.

# Under Rails 7 with 'cssbundling-rails' and/or the 'jsbundling-rails' gems,
# entirely external systems are used for asset management. With Sprockets no
# longer synchronously building assets on-demand and only when the source files
# changed, compiled assets might be (during local development) or will almost
# always be (CI systems) either out of date or missing when tests are run.
#
# People are used to "bundle exec rspec" and things working. The out-of-box gem
# 'cssbundling-rails' hooks into a vanilla Rails "prepare" task, running a full
# "css:build" task in response. This is quite slow and generates console spam
# on every test run, but points to a slightly better solution for RSpec.
#
# This class is a way of packaging that solution. The class wrapper is really
# just a namespace / container for the code.
#
# First, if you aren't already doing this, add the folllowing lines to
# "spec_helper.rb" somewhere *after* the "require 'rspec/rails'" line:
#
#     require 'rake'
#     YourAppName::Application.load_tasks
#
# ...and call MaintainTestAssets::maintain! (see that method's documentation
# for details). See also constants MaintainTestAssets::ASSET_SOURCE_FOLDERS and
# MaintainTestAssets::EXPECTED_ASSETS for things you may want to customise.
#
class MaintainTestAssets

  # All the places where you have asset files of any kind that you expect to be
  # dynamically compiled/transpiled/etc. via external tooling. The given arrays
  # are passed to "Rails.root.join..." to generate full pathnames.
  #
  # Folders are checked recursively. If any file timestamp therein is greater
  # than (newer than) any of EXPECTED_ASSETS, a rebuild is triggered.
  #
  ASSET_SOURCE_FOLDERS = [
    ['app', 'assets', 'stylesheets'],
    ['app', 'javascript'],
    ['vendor']
  ]

  # The leaf files that ASSET_SOURCE_FOLDERS will build. These are all checked
  # for in "File.join(Rails.root, 'app', 'assets', 'builds')". Where files are
  # written together - e.g. a ".js" and ".js.map" file - you only need to list
  # any one of the group of concurrently generated files.
  #
  # In a standard JS / CSS combination this would just be 'application.css' and
  # 'application.js', but more complex applications might have added or changed
  # entries in the "scripts" section of 'package.json'.
  #
  EXPECTED_ASSETS = %w{
    application.js
    application.css
  }

  # Call this method somewhere at test startup, e.g. in "spec_helper.rb" before
  # tests are actually run (just above "RSpec.configure..." works reasonably).
  #
  def self.maintain!
    run_build    = false
    newest_mtime = Time.now - 100.years

    # Find the newest modificaftion time across all source files of any type -
    # for simplicity, timestamps of JS vs CSS aren't considered
    #
    ASSET_SOURCE_FOLDERS.each do | relative_array |
      glob_path = Rails.root.join(*relative_array, '**', '*')

      Dir[glob_path].each do | filename |
        next if File.directory?(filename) # NOTE EARLY LOOP RESTART

        source_mtime = File.mtime(filename)
        newest_mtime = source_mtime if source_mtime > newest_mtime
      end
    end

    # Compile the built asset leaf names into full file names for convenience.
    #
    built_assets = EXPECTED_ASSETS.map do | leaf |
      Rails.root.join('app', 'assets', 'builds', leaf)
    end

    # If any of the source files are newer than expected built assets, or if
    # any of those assets are missing, trigger a rebuild task *and* force a new
    # timestamp on all output assets (just in case build script optimisations
    # result in a file being skipped as "already up to date", which would cause
    # the code here to otherwise keep trying to rebuild it on every run).
    #
    run_build = built_assets.any? do | filename |
      File.exist?(filename) == false || File.mtime(filename) < newest_mtime
    end

    if run_build
      Rake::Task['javascript:build'].invoke()
      Rake::Task[       'css:build'].invoke()

      built_assets.each { | filename | FileUtils.touch(filename, nocreate: true) }
    end
  end
end

(EDIT) As a commenter below points out, you'll need to make sure Rake tasks are loaded in your spec_helper.rb, e.g.:

require 'rake'
Rails.application.load_tasks

Answer 2

Both jsbundling-rails and cssbundling-rails append themselves into a rake task called test:prepare.

There are a few ways to cause test:prepare to run, depending on your overall build process.

1. Call it directly:

   bundle exec rails test:prepare test

   Or, if running rspec outside of the rails command:

   bundle exec rails test:prepare && bundle exec rspec

2. Use a test task that already calls test:prepare.

   Curiously, only some test tasks call (depend on) test:prepare, while others (including the default test task) don't. Example:

   bundle exec rails test:all

3. Make test:prepare a dependency on your preferred test task.

   For example, if you normally use the spec task by running bundle exec rails spec, add this to a new or existing task file (such as lib/tasks/tests.rake):

   task spec: ['css:build', 'javascript:build']

Background

test:prepare is an empty task defined by Rails. Both cssbundling-rails and jsbundling-rails add themselves as dependencies of that task.

In general, test:prepare is a useful place to add any kind of dependency needed to run your tests, with the caveat that only some of Rails' default test tasks depend on it. But as mentioned above, you can always call it directly or add your own dependencies.

In most cases, calling test:prepare is going to be equivalent to calling both css:build and javascript:build, which is why I showed test:prepare in most of the above examples. On occasion, other gems or your app may an extended test:prepare with additional commands as well, in which case those will run as well (and would likely be wanted).

Also note that assets:precompile also depends on css:build and javascript:build. In my experience, test:prepare (or css:build and javascript:build separately) runs faster than assets:precompile, likely because we're running a lightweight configuration of sprockets-rails (as opposed to propshaft) and assets:precompile runs the entire sprockets compilation process.