Pubsync 1.3 Released!

Today I released PubSync 1.3.

PubSync (short for Publish/Sync) is a command-line application I developed to ease the pain of publishing my Visual Studio projects. Although I developed it for use with Visual Studio, it can easily be used for syncing any directory structure.
I created PubSync to solve 3 issues:

1. Visual Studio's Publish feature would often fail to overwrite changed files, forcing me to manually delete the destination files and re-publish. I hate manual steps.
2. Visual Studio's option to remove the files on the destination that don't exist on the source is not configurable. For example, I couldn't choose to have it not wipe out my uploads folder containing user-created content.
3. I sometimes work from home, and publishing through a VPN across the Internet was extremely slow, despite the file sizes being low. In my experience, PubSync is much faster.

PubSync's Main Features:

• Simple XML schema for configuring file sync
• Support for profiles to publish to different servers
• High-speed syncing
• Easily exclude files/folders from syncing using Regular Expressions
• Delete files on the destination that do not exist on the source
• Easily sync a single file

Simple XML schema for configuring file sync

I wanted to make configuring PubSync simple but flexible, so I made up a simple XML schema. The schema and a sample config file are included with PubSync.

Support for profiles to publish to different servers

At work, we use at least separate servers for development: development, staging, and live. I'm all about automating repetitive tasks, so Pubsync needed to support easily publishing to all 3.

High-speed syncing

At the first release, PubSync simply called Robocopy with the proper arguments to perform the sync.

This turned out to be quite a bit faster than Visual Studio, but it was still pretty slow when copying across the VPN from home, so I added a feature in PubSync to perform the sync internally. The Robocopy option is now deprecated and will likely be removed in the near future.

To determine if a file should be copied, PubSync uses a few simple checks:

1. Is the file in the excluded list?
2. Does the file exist on the destination?
3. Do the source and destination files have the same size and date modified?

I'm unsure of all the criteria Robocopy uses to decide what to copy, but my method turned out to be much faster, especially when dealing with many small files.

Delete files on the destination that do not exist on the source

If a file exists on the destination but not on the source, PubSync will remove it. Simple and sweet.

Easily exclude files/folders from syncing using Regular Expressions

I needed PubSync to ignore the user uploads folder, as it only exists on the destination and the files should never be erased. I also required it to skip certain files (all the CSharp files, for example).

PubSync can be configured with regular expressions to exclude certain files and folders.

Easily sync a single file

I was working on a web app across a VPN and got sick of waiting for the entire directory structure to sync (it took about 20 seconds) when I had only changed 1 file, so I added an option to specify a single file to be synced.

My new favorite Visual Studio extension!

Adding GAC references to a Visual Studio project sucks. Why? Because for every reference you add, you have to open the "Add Reference" box, wait 10-20 seconds for it to load the list, find your reference in the list, and click OK. It doesn't cache the list, and it doesn't sort it either (although it does remember your sort preference - if I sort the list A to Z and add a reference, then reopen the Add Reference dialog, the list is unsorted. However, if I then click to sort the list again, the list will be sorted Z to A. Genius! Oh wait...).

When you have to update multiple references in multiple projects, this quickly becomes an aggravating, time consuming process. But no longer! After years of silent suffering, I looked for and found an awesome fix: an VS extension called Muse.VSExtensions (http://visualstudiogallery.msdn.microsoft.com/36a6eb45-a7b1-47c3-9e85-09f0aef6e879/). This extension adds it own dialog for adding GAC references. The references are cached, easily searchable, and can be refreshed with a simple button press.

I only wish I'd found this sooner. Now, back to updating references.

MySQL Failover with Entity Framework

I recently started using the Entity Framework 4.1, and I absolutely love it! It has taken away several of the headaches I've faced; for example, when I have multiple databases with identical schemas, I no longer need to keep separate stored procedures for them - the queries are now handled directly in my code. In addition, I now have strong typing without having to map each DataRow column to a class property. However, I ran into a small snag when it came to server failover. The app I'm working on has 4 MySQL servers - 3 slaves, 1 master. Two of the slaves are in a different site, so they are used only if the main read and master servers go offline. In the old version of the app, I used a custom solution to direct the queries to the proper servers. For example, instead of using this code:

var datatable = new DataTable();
using(var command = new MySqlCommand("Render_GetGraphics", new MySqlConnection("connectionstring")))
{
    command.Parameters.AddRange(new MySqlParameterCollection[]
                                    {
                                        new MySqlParameter("_clientId", _clientId),
                                        new MySqlParameter("_otherId", _otherId),
                                    });
    datatable.Load(command.ExecuteReader());
}
var graphics = datatable.Rows.Cast<DataRow>().Select(
        r =>
        new
        {
            Name = (string)r["Name"],
            Action = (GraphicApplicationAction)r["ActionID"],
            AlwaysApply = Convert.ToBoolean(r["AlwaysApply"]),
            PageSettings = (string)r["PageSettings"],
            IsEnabled = Convert.ToBoolean(r["IsEnabled"]),
            Image = r.IsNull("Image") ? null : (byte[])r["Image"],
            Type = r.IsNull("Type") ? "" : (string)r["Type"]
        })
    .ToList();

I used this code:

public ConnectionInfo dbConnection = new ConnectionInfo(null, new [] { "Server1", "Server2" }, "dbname", "dbuser",
                                                                   "dbpassword");
var graphics = new DbCommand(dbConnection, "Render_GetGraphics",
                                         new[]
                                             {
                                                 new MySqlParameter("_clientId", _clientId),
                                                 new MySqlParameter("_otherId", _otherId),
                                             })
                .RetrieveDataRows().Select(
                    r =>
                    new
                        {
                            Name = (string)r["Name"],
                            Action = (GraphicApplicationAction)r["ActionID"],
                            AlwaysApply = Convert.ToBoolean(r["AlwaysApply"]),
                            PageSettings = (string)r["PageSettings"],
                            IsEnabled = Convert.ToBoolean(r["IsEnabled"]),
                            Image = r.IsNull("Image") ? null : (byte[])r["Image"],
                            Type = r.IsNull("Type") ? "" : (string)r["Type"]
                        })
                .ToList();

The RetrieveDataRows() method had error handling so that if it had trouble connecting to one server, it could try the next best option. With the EntityFramework, I no longer had that option, as my old model would have prevented me from using LINQ efficiently. So I spent this morning working on it, and came up with these two classes: EntityFrameworkConnection.cs

using System;
using System.Collections.Generic;
using System.Data.EntityClient;
using System.Linq;
using MySql.Data.MySqlClient;

namespace myApp
{
    public class EntityFrameworkConnection
    {
        # region Fields

        readonly List<string> _servers = new List<string>();
        private readonly string _initialConnectionString;
        # endregion

        # region Properties

        public string ConnectionString
        {
            get
            {
                lock (_servers)
                {
                    EntityConnectionStringBuilder entityBuilder = new EntityConnectionStringBuilder(_initialConnectionString);
                    entityBuilder.ProviderConnectionString = String.Format(entityBuilder.ProviderConnectionString, "localhost");
                    MySqlConnectionStringBuilder sqlConnectionBuilder = new MySqlConnectionStringBuilder(entityBuilder.ProviderConnectionString) { Server = _servers.First() };
                    entityBuilder.ProviderConnectionString = sqlConnectionBuilder.ToString();

                    return entityBuilder.ToString();
                }
            }
        }

        public List<string> Servers { get { lock (_servers) return _servers; } }

        # endregion

        # region Constructors

        public EntityFrameworkConnection(string connectionString)
        {
            _initialConnectionString = connectionString;
        }

        # endregion

        # region Public Methods

        public bool GoToNextServer()
        {
            lock (_servers)
                if (_servers.Any())
                    _servers.RemoveAt(0);

            return _servers.Any();
        }

        # endregion
    }
}

The final version of this class will be able to update the list of servers from the database. This version has to have the servers manually specified. After a failed connection to a server, that server is removed from the list of available servers, and the next server in the List<> is used. AdminDb.cs

using System;
using System.Configuration;
using System.Linq;
using MySql.Data.MySqlClient;

namespace myApp
{
    public class AdminDb
    {
        public delegate T QueryDelegate<T>(Models.adminEntities db);

        # region Instance Stuff
        private readonly EntityFrameworkConnection _dbConnection;
        //public EntityFrameworkConnection DbConnection { get { return _dbConnection; } }
        
        private AdminDb()
        {
            _dbConnection = new EntityFrameworkConnection(
                ConfigurationManager.ConnectionStrings["adminEntities"].ConnectionString);
        }

        # endregion 

        # region Static Stuff
        private static AdminDb _manager;

        public static EntityFrameworkConnection Connection { get { return _manager._dbConnection; } }
        
        static AdminDb()
        {
            _manager = new AdminDb();
            // Manually add the servers
            Connection.Servers.Add("localhost");
        }

        public static T RunQuery<T>(QueryDelegate<T> del, bool isReadonly = true)
        {
            while (_manager._dbConnection.Servers.Any())
            {
                try
                {
                    using (
                        var db =
                            new Models.adminEntities(
                                _manager._dbConnection.ConnectionString))
                    {
                        return del(db);
                    }
                }
                catch (Exception ex)
                {
                    if (ex.InnerException is MySqlException &&
                        ex.Message == "Unable to connect to any of the specified MySQL hosts.")
                        if (_manager._dbConnection.GoToNextServer())
                            continue;

                    throw;
                }
            }

            return default(T);
        }
        # endregion
    }
}

This class acts as a layer between Entity Framework and my app, enabling me to easily do things like failover while still keeping the simplicity of EntityFramework. For each EF model I simply have to create a version of this class that uses the proper connection string and model. Example time:

 
// Standard entity framework call:
using (var db = new estatements_adminEntities())
{
    var usersList = db.users.Include(u => u.clientgroup).Include(u => u.client).ToList();
    // do stuff..
}

// The same call using AdminDb:
var usersList = AdminDb.RunQuery(db => db.users.Include(u => u.clientgroup).Include(u => u.client)).ToList();

Serializing dates in MVC (update)

As I was falling asleep in bed last night, a thought went through my mind: "Since I'm using this Regular Expression every time I load the page, isn't it rather wasteful to be compiling it every time?" So when I got up this morning, I looked up "Compiled Regular Expressions", and read through this page: http://www.dijksterhuis.org/regular-expressions-advanced/. Not only did I pick up some tips on compiling regular expressions, I also discovered a much simpler (and likely quite more efficient) way of replacing the MS-formatted date with my date string. In my previous method, I was casting the Regex matches to an IENumerable and using Linq to iterate through them; in the updated version, the iterative process is handled internally by the Regex object. All I have to do is tell it to use my FixDates method whenever it encounters a match. In the process of updating my code, I also added support for custom Date Formats and set the default to "yyyy-MM-dd".

private static readonly Regex DateRegex = new Regex(@"\\/Date\((?<ticks>\d+)?\)\\/", RegexOptions.Compiled);
/// <summary>
/// If true, date-specific serialization code will be run;
/// If false, date-specific serialization code will not be run.
/// </summary>
public bool HasDates { get; set; }
/// <summary>
/// The date format string to be applied. Defaults to "yyyy-MM-dd".
/// </summary>
public string DateFormatString { get; set; }

/// <summary>
/// Replaces MS-formatted dates with date strings
/// </summary>
/// <param name="match"></param>
/// <returns></returns>
public string FixDates(Match match)
{
    return new DateTime(1970, 1, 1)
        .AddMilliseconds(long.Parse(match.Groups["ticks"].Value))
        .ToString(DateFormatString);
}

/// <summary>
/// Serialize the object to JavaScript and
/// perform extra formatting on the serialized string as necessary
/// </summary>
/// <returns></returns>
public string Serialize()
{
    JavaScriptSerializer serializer = new JavaScriptSerializer();
    var serializedData = serializer.Serialize(Data);
    if (HasDates)
        serializedData = DateRegex.Replace(serializedData, FixDates);

    return serializedData;
}

Click here to download the updated code.

Serializing dates in MVC

Update: I've rewritten the code for simplicity and efficiency. You can check it out here.
Ah, dates. Simple little things, really, especially when working with .Net and JavaScript, right?
Well, sort of. See, Microsoft decided that when serializing DateTime objects, they should be formatted like this:
"\/Date(12345678)\/". Unfortunately, without running eval() on that, that format is useless.
I've never really had this issue before, because when working with dates on the client side, I always use strings.
So why the issue now?
Because, along with using MVC, I decided to use the Entity Framework, which made it really easy to return a bunch of objects from the database:

return Json(db.Tasks.ToList());

In the past, this would have been more like:

DataTable dt = new DataTable();
dt.Load(command.ExecuteReader());
return dt.Rows.Cast<DataRow>().Select(
                r =>
                new
                    {
                        id = (int) r["id"],
                        name = (string) r["name"],
                        date = ((DateTime) r["date"]).ToString("yyyy-MM-dd")
                    });

Now, I could simply go on using the select statement, but I just couldn't bear the thought of doing that everytime I had to work with dates (since that is all the time). So instead I wrote a custom Json serializer derived from JsonResult. I've uploaded the source code (link at bottom), but the real magic is in these lines:

public bool HasDates { get; set; }

public override void ExecuteResult(ControllerContext context)
{ 
    //... a bunch of code ...
    if (Data != null)
    {
        JavaScriptSerializer serializer = new JavaScriptSerializer();
        response.Write(HasDates ? FixDates(serializer.Serialize(Data)) : serializer.Serialize(Data));
    }
}   

public string FixDates(string data)
{
    var matches = Regex.Matches(data, @"\\/Date\((?<ticks>\d+)?\)\\/").Cast<Match>().ToArray();
    for (int i = matches.Length - 1; i >= 0; i--)
    {
        var match = matches[i];
        data = data.Remove(match.Index, match.Length)
            .Insert(match.Index,
                    new DateTime(1970, 1, 1).AddMilliseconds(long.Parse(match.Groups["ticks"].Value))
                        .ToString("yyyy-MM-dd HH:mm:ss"));
    }
    return data;
}

The code simply loops through the standard serialized result and replaces any MS-formatted date strings with my kind of date string (yyyy-MM-dd HH:mm:ss eg. "2011-11-21 19:48:13").

Updated return call:

return new ProperJsonResult() { HasDates = true, Data = db.Tasks.ToList() };

Click here to download ProperJsonResult.cs

Rectangle.ReallyContains()

So I just wrote an extension method, Rectangle.ReallyContains(Point p).
Seems the XNA folks and I disagree on what Contains() really means. For example, let's say a rectangle has a left value of 0, a top value of 0, a right value of 5, and a bottom value of 5.
To me, the corners of this rectangle are as follows: (0,0), (0,5), (5,5), (5,0).
So then, rectangle.Contains(new Point(0,5)) should be true.
According to the XNA folks, this is not correct. From what I can tell, the bottom and right values are the numbers directly after the rectangle ends (so the rectangle goes from 0 to - but not including - 5).
I'm sure their way makes sense in a lot of scenarios. It doesn't make sense for mine, and I just spent a good 10 minutes figuring that out.
*rant finished*

Nate's Roguelike Game

So I recently rediscovered ADOM (http://www.adom.de/) and after playing it for a while (and dying many times) I've decided to create my own roguelike game.
The game is located here: http://natesroguelikegame.codeplex.com/.
There are no releases as yet - so far, the only thing working is random room generation.