Reinventing the System.Web.Optimization wheel with gulp

For a long time, I was using the standard bundling and minification System.Web.Optimization framework to deliver bundled and minified JavaScript and CSS to users of our app. In one of Azure's more frustrating changes, the 'In Role' cache was removed from the Cloud Service offering. The only cache choice now is Redis. Which is great, but it caused us some infrequent problems with bundles getting "stuck". We'd have to run a console application to clear out all of the bundle information in Redis whenever that happened.

We had need for a better front end build anyway as we started to explore replacements for angular (e.g. Aurelia) so decided to add bespoke bundling and minification. The actual implementation is more complex than what I'll show here because the code runs 2 'brands' (sites), each in 3 different countries. Let's assume the code runs one brand in one deployment instead!

Config

The bundles are configured in a file called "bundles.js". This file exports a single array of objects each defining a bundle.

 module.exports = [  
   {  
     "name": "primary-bundle.js",   
     "files": [  
       "content/js/file1.js",   
       "content/js/file2.js"]  
   },  
   {  
     "name": "primary-bundle.css",  
     "files": [  
       "content/css/file1.css",  
       "content/css/file2.css"  
     ]  
   }  
 ]

This defines a primary-bundle.js JavaScript bundle containing two files and a similar CSS bundle.

Gulp file

The gulp file takes this configuration and bundles the files and minifies them. It looks like this:

var gulp = require("gulp"),  
   concat = require("gulp-concat"),  
   cssmin = require("gulp-cssmin"),  
   uglify = require("gulp-uglify"),  
   merge = require("merge-stream"),  
   bundles = require('../bundles.js'),  
   paths = require('../paths'),  
   rev = require('gulp-rev'),  
   footer = require('gulp-footer'),  
   rename = require('gulp-rename'),  
   cleanCss = require('gulp-clean-css');  

 var regex = {  
   css: /\.css$/,  
   js: /\.js$/  
 };  

 gulp.task("bundle-all", ["bundle"], function() {  
   var jsTasks = getBundles(regex.js).map(function (bundle) {  
     return gulp.src(bundle.files)  
       .pipe(footer(";"))  // Ensures scripts are isolated
       .pipe(concat(bundle.name))  
       .pipe(uglify())  
       .pipe(rev())  // adds a revision to the file name
       .pipe(gulp.dest(paths.bundleOutput));  
   });  
   var cssTasks = getBundles(regex.css).map(function (bundle) {  
     return gulp.src(bundle.files) // rebases the relative paths in the css files to the output directory.  
       .pipe(cleanCss({  
           rebaseTo: process.cwd() + "/" + paths.bundleOutput  
       }))  
       .pipe(concat(bundle.name))  
       .pipe(rev())  // Adds a revision to the file name
       .pipe(gulp.dest(paths.bundleOutput));  
   });  
   return merge(jsTasks)  
     .add(cssTasks)  
     .pipe(rev.manifest())  // Generates a manifest file
     .pipe(gulp.dest(paths.output));  
 });  
 function getBundles(regexPattern) {  
   return bundles.filter(function(bundle) {  
     return regexPattern.test(bundle.name);  
   });  
 }

There's another file "paths.js" which just has a bunch of properties for the paths to interesting places. E.g. source file locations and the output directories.

This task bundles both the css and javascript files and uses the gulp-rev plugin to add the revision to the file name. The manifest file this creates is then parsed later when rendering bundles in the HTML.

Rendering the bundles to the HTML

The MVC views have no knowledge of what goes on. They simply use an HTML helper to render the tags:

public static class Assets  
 {  
   private static readonly string ScriptFormat = "<script src=\"{0}\"></script>";  
   private static readonly string StyleFormat = "<link href=\"{0}\" rel=\"stylesheet\"/>";  
   public static IHtmlString RenderScript(string path)  
   {  
     var bundlePath = $"{path}.js";  
     return new HtmlString(  
       string.Join(  
         Environment.NewLine,   
         BundleManager.GetRenderableAssetsForBundle(bundlePath)  
           .Select(f => string.Format(ScriptFormat, f))));  
   }  
   public static IHtmlString RenderStyle(string path)  
   {  
     var bundlePath = $"{path}.css";  
     return new HtmlString(  
       string.Join(  
         Environment.NewLine,  
         BundleManager.GetRenderableAssetsForBundle(bundlePath)  
           .Select(f => string.Format(StyleFormat, f))));  
   }  
 }
Used like this:

 @Assets.RenderScript("primary-bundle")

This helper uses the BundleManager to get a list of files:

 public static class BundleManager  
 {  
     private static readonly object Locker = new object();  
     private static IBundleFileMapper fileMapper;  
     public static IEnumerable<string> GetRenderableAssetsForBundle(string bundlePath)  
     {  
         return fileMapper.GetBundleFiles(bundlePath);  
     }  
     public static void Initialize(string manifestPath, string bundleConfigPath)  
     {  
         if (fileMapper != null)  
         {  
             return;  
         }  
         lock (Locker)  
         {  
             if (fileMapper != null)  
             {  
                 return;  
             }  
             #if DEBUG  
                 fileMapper = IndividualBundleFileMapper.Create(bundleConfigPath);  
             #else  
                 fileMapper = VersionedBundleFileMapper.Create(manifestPath);  
             #endif  
         }  
     }  
 }

Which in turn uses an IBundleFileMapper:

 public interface IBundleFileMapper  
 {  
   IEnumerable<string> GetBundleFiles(string bundleName);  
 }

To avoid this becoming a code dump, I'll just show the VersionedBundleFileMapper:

public class VersionedBundleFileMapper : IBundleFileMapper  
 {  
     private const string BundleFolder = "/dist/bundles/";  
     private VersionedBundleFileMapper()  
     {  
     }  
     private IDictionary<string, string> currentMappings;  
     public IEnumerable<string> GetBundleFiles(string bundleName)  
     {  
         if (string.IsNullOrWhiteSpace(bundleName))  
         {  
             throw new ArgumentException($"Unknown file name in ${bundleName}");  
         }  
         if (!currentMappings.TryGetValue(bundleName, out string versionedFileName))  
         {  
             throw new ArgumentException("${bundleName} is not a known bundle.");  
         }  
         return new[] { $"{BundleFolder}{versionedFileName}" };  
     }  
     public static VersionedBundleFileMapper Create(string path)  
     {  
         string config = File.ReadAllText(path);  
         return new VersionedBundleFileMapper  
         {  
             currentMappings =  
                 JsonConvert.DeserializeObject<ConcurrentDictionary<string, string>>(config)  
         };  
     }  
 }

This class loads the config at the supplied path. This is the rev-manifest json file created in the gulp task. The location of this file is passed in when the BundleManager is configured (at app start up):

BundleManager.Initialize(  
   Server.MapPath("~/dist/rev-manifest.json"),   
   Server.MapPath("~/gulpBuild/bundles.js"));

The rev-manifest.json is just a map of files with the revision in the name to the raw file name.

Comments

Post a Comment

Popular posts from this blog

Trimming strings in action parameters in ASP.Net Web API

In standard MVC it's very easy to create a model binder to trim all of the strings passed into action methods. The approach I used for that was from here:  http://blogs.taiga.nl/martijn/2011/09/29/custom-model-binders-and-request-validation/ . It works well and it trims the properties of models too. The problem came when I tried to do the same thing for Web API. I added a similar StringTrimmingBinder and registered it as a service using the very non-discoverable API: var provider = new SimpleModelBinderProvider( typeof ( string ),  new StringTrimmingModelBinder()); GlobalConfiguration.Configuration.Services.Insert( typeof (ModelBinderProvider),  0,  provider); As a side note, I have no idea how you're supposed to find out how to register a model binder without opening up Bing and Googling it. Anyway, this worked great when the parameters to the action were actually strings but doesn't work with complex types as parameters. Now, I'v...
Read more

Full text search in Entity Framework 6 using command interception

Image
Background One of the main problems we encountered using Entity Framework was its lack of support for full text search. Our search function is based around a series of processors that attempt to match the incoming phrase against a set of patterns that we're expecting (10+). For example, there's one for dates and times, recent financial quarters as well as more domain specific ones. Everything is cached and so is really fast, except when it isn't: Here's the monitoring chart for our SQL DB in Azure That's a spike to 100% resource utilisation because we fall through to a phrase search as a last resort (which cannot be cached) - it's exceedingly rare but it still hurts. Before EF, the app used full text search and it was one of the casualties when the app was migrated to EF4 several years ago. The lack of full text search has been getting more and more painful as the amount of data has grown. We really needed full text search back... Command intercept...
Read more

Composing Expressions in C#

This post follows on from  Reusing predicates in Entity Framework . In my previous post I introduced the "and" extension method which allowed me to create a new entity framework safe expression from two existing expressions. Here's how we make that happen: The code is from a blogpost from Colin Meek from 2008:  InvocationExpression and LINQ to Entities . I'll just dump my version of the code here. The idea is exactly the same (even using the visitor as an immutable stack): public class InvocationExpander : ExpressionVisitor { private readonly ParameterExpression _parameter; private readonly Expression _expansion; private readonly InvocationExpander _previous; public static T Expand<T>(T expr) where T : LambdaExpression { return (T) new InvocationExpander().Visit(expr); } public InvocationExpander() { } private InvocationExpander(ParameterExpression parameter, Expression expansion, Invocati...
Read more