Techie Head

For a long time, I was using the standard  bundling and minification  System.Web.Optimization framework to deliver bundled and minified JavaScript and CSS to users of our app. In one of Azure's more frustrating changes, the 'In Role' cache was removed from the Cloud Service offering. The only cache choice now is Redis. Which is great, but  it caused us some infrequent problems with bundles getting "stuck". We'd have to run a console application to clear out all of the bundle information in Redis whenever that happened. We had need for a better front end build anyway as we started to explore replacements for angular (e.g. Aurelia) so decided to add bespoke bundling and minification. The actual implementation is more complex than what I'll show here because the code runs 2 'brands' (sites), each in 3 different countries. Let's assume the code runs one brand in one deployment instead! Config The bundles are configured in a file called ...

This post follows on from  Reusing predicates in Entity Framework . In my previous post I introduced the "and" extension method which allowed me to create a new entity framework safe expression from two existing expressions. Here's how we make that happen: The code is from a blogpost from Colin Meek from 2008:  InvocationExpression and LINQ to Entities . I'll just dump my version of the code here. The idea is exactly the same (even using the visitor as an immutable stack): public class InvocationExpander : ExpressionVisitor { private readonly ParameterExpression _parameter; private readonly Expression _expansion; private readonly InvocationExpander _previous; public static T Expand<T>(T expr) where T : LambdaExpression { return (T) new InvocationExpander().Visit(expr); } public InvocationExpander() { } private InvocationExpander(ParameterExpression parameter, Expression expansion, Invocati...

Warning: some knowledge of Expressions  is required. Entity Framework simplifies a great number of things; for starters, I don't have to write a lot of SQL which makes development a lot faster - I'm a lot more productive in C# than SQL. One thing that works really well in EF, is the ability to reuse predicates - how often do you find yourself typing: posts.Where(p => !p.IsDeleted); In progamming we often get taught to DRY out the code - i.e. Don't Repeat Yourself. I think that extends to the expressions we're pass in to EF. The expressions are the specification language of EF (ignore that part if you don't do DDD). I'll look at a couple of ways of storing that expression - first up: extension methods. Set up Let's a assume we have a blog post entity like: public class BlogPost { public int Id { get ; set ; } public string Title { get ; set ; } public string Body { get ; set ; } public string AuthorName { get ; set ; } ...

I wanted to persist some extra claims that weren't driven directly from the database. That was all well and good until the user identity was refreshed and the extra claims were lost. The template code for Startup.cs looks like this: app.UseCookieAuthentication( new CookieAuthenticationOptions { AuthenticationType = DefaultAuthenticationTypes.ApplicationCookie, LoginPath = new PathString( "/Account/Login" ), Provider = new CookieAuthenticationProvider { // Enables the application to validate the security stamp when the user logs in. // This is a security feature which is used when you change a password or add an external login to your account. OnValidateIdentity = SecurityStampValidator.OnValidateIdentity<ApplicationUserManager, ApplicationUser>( validateInterval: TimeSpan.FromMinutes(30), regenerateIdentity: (manager, user) => user.GenerateUserIdentityAsync(manager)) } }); It tur...

When I write and review code I'm always looking to have one thing happen per line. I've mostly favoured this because of how much more readable it makes the code e.g. see this review http://codereview.stackexchange.com/a/111706/22816. However, I refactored some code and introduced a massive bug (caught by tests!) because I missed a side effect... The real code was a lot longer (100+ LoC method - hence the refactor). It was something of this form though: var i = 0; var maxAttempts = 5; while ( true ) { var workRequest = someQueue.GetWork(); var workResult = someService.Execute(workRequest); if (!workResult.IsSuccess) { Log( "{0}, {1}, {2}" , workRequest.Name, i++, workResult.ErrorsDisplay); } if (i > maxAttempts) { throw new DomainException( "Some details!" ); } } When I refactored the code I changed the line that incremented the  i  variable and lost the side effect! It's a really...

All of our decisions when creating software have pros and cons. Solution A might solve the problem well but lack flexibility. Solution B might have loads of flexibility and solve the problem just as well but be difficult to understand. A third solution, C, might be flexible and intuitive but involve a large upfront cost. I remember when I first started developing software, I'd be presented with a problem and then I'd work at it until I solved it and then move on. It was certainly a simpler time! The issue now is that I see so many different ways of solving a problem it can be almost impossible to choose which one to go with. There are several common themes I think about when evaluating the options: Simplicity Flexibility Testability However, aside from the technical considerations there's also the business concerns: Deadline Severity Budget It's these concerns which generally lead to technical debt . It's a great metaphor. Just as financial debt ...

Background One of the main problems we encountered using Entity Framework was its lack of support for full text search. Our search function is based around a series of processors that attempt to match the incoming phrase against a set of patterns that we're expecting (10+). For example, there's one for dates and times, recent financial quarters as well as more domain specific ones. Everything is cached and so is really fast, except when it isn't: Here's the monitoring chart for our SQL DB in Azure That's a spike to 100% resource utilisation because we fall through to a phrase search as a last resort (which cannot be cached) - it's exceedingly rare but it still hurts. Before EF, the app used full text search and it was one of the casualties when the app was migrated to EF4 several years ago. The lack of full text search has been getting more and more painful as the amount of data has grown. We really needed full text search back... Command intercept...

I've been a fan of Application Insights for quite some time now, it's helped debug and resolve problems that we didn't even know we had! In the new tiers in Azure, the free tier gets 5 million data points a month. That sounds like a lot, until you get a message 10 days into the month warning that you've already used 80% of the limit. Our problem was that it tracks a whole heap of requests to things we don't care about. For example, requests for images, some AJAX requests for social media sharing links and our custom health probe (used by Azure load balancing to decide whether to route requests) were taking up a big portion of that data allowance. I opened up Bing and Googled whether I could exclude some urls - that led me to this Stack Overflow question: http://stackoverflow.com/questions/31149720/is-it-possible-to-exclude-a-url-from-application-insights It looked sensible enough. I first upgraded all of my Application Insights NuGet packages to the latest ver...

I just had a problem that has taken ages to solve so blogging on the off chance it helps someone in need (or possibly future me). When this happens you need to check that you don't have a dodgy instance lying around on your localdb. All of the stuff I read online pointed at the storage emulator using (localdb)\v11.0 but I couldn't see anything in there when I checked. I decided to do a bit more digging: Running sqllocaldb i lists the instances on my localdb... I connected to MSSQLLocalDB with Visual Studio's Sql Server Object Explorer (Sql Server Management Studio would work just as well) And deleted the AzureStorageEmulatorDb* entries. The next time I started to storage emulator it all worked fine!

Froala was very straightforward to get up and running. Their documentation is pretty good but lacks an English version of the server integration for image/file uploads. Ignoring the paste, here's a view of the simple upload of images/files. I have added a parameter to the method called location which lets me have the editor save files to different places depending on where the editor is. [HttpPost] [Route( "content/uploadfile" )] public ActionResult UploadFile( string location, HttpPostedFileBase file) { string fileName; try { // resourceService handles resizing image and saving to Azure blob storage. fileName = resourceService.AddFileInLocation(location, file); } catch (Exception ex) { return Json2( new { error = ex.Message }); } // /resources/ proxies requests to assets var data = new { link = ( "/resources/" + location + "/" + fileName).ToLower() }; return Json2(d...

When needing to add a bit of WYSIWYG into a web application I've always gone with the stalwart TinyMCE. It does the job, it's configurable with regards to what you allow in your elements and generally users use it without too much trouble. We just had a few issues that were annoying with TinyMCE so we went on a hunt for a new editor to use in its place. A colleague recommended Froala  as he had recently evaluated it for another project. We use angular extensively (and had to write a directive for TinyMCE integration) so I was expecting to have to do the same for Froala. Not so! They already have a directive for angular available on GitHub . It took approximately 5 minutes to go from TinyMCE to basic Froala. It took another day or so to get all the functionality we wanted - that includes creating APIs for image and file upload. Things I really like about Froala: Good documentation It seems to produce better html than TinyMCE did (fewer random <p> tags!) Pastin...

Today I messed up. I'm a strong believer in sharing experiences of failure. If no one talks about the times they make mistakes, we're going to be doomed to repeat the same errors over and over again. So, here goes... Today I made a big mistake in a production application that resulted in 15 minutes of frantic database scripting and partial service interruption. I'm going to share what I did wrong, in part so that I focus on what I can do better next time but also because I don't want other people to make the same mistakes. Here's what happened. In our application we have some entities which use a natural key rather than a database generated primary key. This can make life easier when querying as you reduce the number of joins but still have referential integrity (with the cost of a bit of denormalisation). In the past, the whole DAL was hand coded SQL so it saved a considerable amount of developer time; we now use an ORM so we really don't care about the ...

In our application we have multiple web front ends communicating with one or more worker roles. This allows our worker roles to take the brunt of expensive operations. For example, some (very infrequent) actions require up to about 500 emails being sent (via Amazon SES). Before Azure (BA), we did everything on the web server - it wasn't common but sometimes users reported over a minute from clicking the button to the result being displayed. Our move to using workers responding to queue messages has massively sped this sort of thing up. Most actions on the worker are triggered by queue messages but we do have several things that just need to happen at set intervals (we use blob leases to prevent multiple workers doing the same thing). While implementing the "ExclusiveJobHost" and "BlobLease" classes, I noticed a common pattern that I ended up pulling out: public class RecurringAction { private ManualResetEvent exitSignal = new ManualResetEvent( false ...

Microsoft Azure Table storage is a very flexible No-Sql store for structured data. Each entity in the table is uniquely identified by a partition key and a row key. A good choice of these is beyond the scope of this post. However, it can be useful to check that entity you want to add to the table has valid partition and row keys. Here's a simple class that encapsulates those rules: /// <summary> /// Encapsulates knowledge about valid values for keys in Microsoft Azure table storage. /// </summary> /// <remarks> /// Rules are found here http://msdn.microsoft.com/library/azure/dd179338.aspx /// </remarks> internal static class KeyValidator { /// <summary> /// Determines whether the specified key is valid. /// </summary> /// <param name="keyValue">The key.</param> /// <param name="errors">Any errors in the value supplied.</param> /// <returns><c>tru...

Like most developers I've experimented with lots of different ways of managing bugs, feature requests, architectural notes and documentation. I always start a new project determined to use a really air tight system for logging bugs and feature requests. I also try to create a central, accessible store for documentation which is searchable and easy for everyone to use. You know what? It never worked. I used a desktop "help desk" style system that logged absolutely everything but had no simple overview so figuring out what status a ticket was in involved opening it and reading through the history. Discussions were often via email and the help desk system wasn't updated. I'd get so fed up with spending 30 minutes setting up a ticket in the system that I'd resort to "todo" lists scattered over my desk. I eventually moved on to OneNote and finally Trello for my own time management although the department was officially using the desktop app. I moved...

I absolutely hate password "strength" plugins - they're almost always terrible. I use a password manager so all my passwords are generated according to the rules of the website I'm on to be as strong as the site will allow (within reason). So I was on NPower trying to register, it told me up to 20 chars, mixed case and at least one number. I got my password manager to generate me a 20 character mixed case password with numbers. I forgot to tick the box for special characters. The results for strength of password were alarming:  . @InfoSecInsanity nearly chose a bad password for @npowerhq but thanks to the PW strength test "Strong" now #FAIL pic.twitter.com/aLyCU184Q6 — Robert Head (@RobHead89) August 29, 2014 So 20 random chars is the same strength as "Orange1"? Add a space to the end (or any special character) and it's suddenly stronger? Who writes these things anyway? Validating the strength of a password is really bloody ...

In standard MVC it's very easy to create a model binder to trim all of the strings passed into action methods. The approach I used for that was from here:  http://blogs.taiga.nl/martijn/2011/09/29/custom-model-binders-and-request-validation/ . It works well and it trims the properties of models too. The problem came when I tried to do the same thing for Web API. I added a similar StringTrimmingBinder and registered it as a service using the very non-discoverable API: var provider = new SimpleModelBinderProvider( typeof ( string ),  new StringTrimmingModelBinder()); GlobalConfiguration.Configuration.Services.Insert( typeof (ModelBinderProvider),  0,  provider); As a side note, I have no idea how you're supposed to find out how to register a model binder without opening up Bing and Googling it. Anyway, this worked great when the parameters to the action were actually strings but doesn't work with complex types as parameters. Now, I'v...