Thursday, December 8, 2011

MySQL Failover with Entity Framework

I recently started using the Entity Framework 4.1, and I absolutely love it! It has taken away several of the headaches I've faced; for example, when I have multiple databases with identical schemas, I no longer need to keep separate stored procedures for them - the queries are now handled directly in my code. In addition, I now have strong typing without having to map each DataRow column to a class property. However, I ran into a small snag when it came to server failover. The app I'm working on has 4 MySQL servers - 3 slaves, 1 master. Two of the slaves are in a different site, so they are used only if the main read and master servers go offline. In the old version of the app, I used a custom solution to direct the queries to the proper servers. For example, instead of using this code:
var datatable = new DataTable();
using(var command = new MySqlCommand("Render_GetGraphics", new MySqlConnection("connectionstring")))
{
    command.Parameters.AddRange(new MySqlParameterCollection[]
                                    {
                                        new MySqlParameter("_clientId", _clientId),
                                        new MySqlParameter("_otherId", _otherId),
                                    });
    datatable.Load(command.ExecuteReader());
}
var graphics = datatable.Rows.Cast<DataRow>().Select(
        r =>
        new
        {
            Name = (string)r["Name"],
            Action = (GraphicApplicationAction)r["ActionID"],
            AlwaysApply = Convert.ToBoolean(r["AlwaysApply"]),
            PageSettings = (string)r["PageSettings"],
            IsEnabled = Convert.ToBoolean(r["IsEnabled"]),
            Image = r.IsNull("Image") ? null : (byte[])r["Image"],
            Type = r.IsNull("Type") ? "" : (string)r["Type"]
        })
    .ToList();
I used this code:
public ConnectionInfo dbConnection = new ConnectionInfo(null, new [] { "Server1", "Server2" }, "dbname", "dbuser",
                                                                   "dbpassword");
var graphics = new DbCommand(dbConnection, "Render_GetGraphics",
                                         new[]
                                             {
                                                 new MySqlParameter("_clientId", _clientId),
                                                 new MySqlParameter("_otherId", _otherId),
                                             })
                .RetrieveDataRows().Select(
                    r =>
                    new
                        {
                            Name = (string)r["Name"],
                            Action = (GraphicApplicationAction)r["ActionID"],
                            AlwaysApply = Convert.ToBoolean(r["AlwaysApply"]),
                            PageSettings = (string)r["PageSettings"],
                            IsEnabled = Convert.ToBoolean(r["IsEnabled"]),
                            Image = r.IsNull("Image") ? null : (byte[])r["Image"],
                            Type = r.IsNull("Type") ? "" : (string)r["Type"]
                        })
                .ToList();
The RetrieveDataRows() method had error handling so that if it had trouble connecting to one server, it could try the next best option. With the EntityFramework, I no longer had that option, as my old model would have prevented me from using LINQ efficiently. So I spent this morning working on it, and came up with these two classes: EntityFrameworkConnection.cs
using System;
using System.Collections.Generic;
using System.Data.EntityClient;
using System.Linq;
using MySql.Data.MySqlClient;

namespace myApp
{
    public class EntityFrameworkConnection
    {
        # region Fields

        readonly List<string> _servers = new List<string>();
        private readonly string _initialConnectionString;
        # endregion

        # region Properties

        public string ConnectionString
        {
            get
            {
                lock (_servers)
                {
                    EntityConnectionStringBuilder entityBuilder = new EntityConnectionStringBuilder(_initialConnectionString);
                    entityBuilder.ProviderConnectionString = String.Format(entityBuilder.ProviderConnectionString, "localhost");
                    MySqlConnectionStringBuilder sqlConnectionBuilder = new MySqlConnectionStringBuilder(entityBuilder.ProviderConnectionString) { Server = _servers.First() };
                    entityBuilder.ProviderConnectionString = sqlConnectionBuilder.ToString();

                    return entityBuilder.ToString();
                }
            }
        }

        public List<string> Servers { get { lock (_servers) return _servers; } }

        # endregion

        # region Constructors

        public EntityFrameworkConnection(string connectionString)
        {
            _initialConnectionString = connectionString;
        }

        # endregion

        # region Public Methods

        public bool GoToNextServer()
        {
            lock (_servers)
                if (_servers.Any())
                    _servers.RemoveAt(0);

            return _servers.Any();
        }

        # endregion
    }
}
The final version of this class will be able to update the list of servers from the database. This version has to have the servers manually specified. After a failed connection to a server, that server is removed from the list of available servers, and the next server in the List<> is used. AdminDb.cs
using System;
using System.Configuration;
using System.Linq;
using MySql.Data.MySqlClient;

namespace myApp
{
    public class AdminDb
    {
        public delegate T QueryDelegate<T>(Models.adminEntities db);

        # region Instance Stuff
        private readonly EntityFrameworkConnection _dbConnection;
        //public EntityFrameworkConnection DbConnection { get { return _dbConnection; } }
        
        private AdminDb()
        {
            _dbConnection = new EntityFrameworkConnection(
                ConfigurationManager.ConnectionStrings["adminEntities"].ConnectionString);
        }

        # endregion 

        # region Static Stuff
        private static AdminDb _manager;

        public static EntityFrameworkConnection Connection { get { return _manager._dbConnection; } }
        
        static AdminDb()
        {
            _manager = new AdminDb();
            // Manually add the servers
            Connection.Servers.Add("localhost");
        }

        public static T RunQuery<T>(QueryDelegate<T> del, bool isReadonly = true)
        {
            while (_manager._dbConnection.Servers.Any())
            {
                try
                {
                    using (
                        var db =
                            new Models.adminEntities(
                                _manager._dbConnection.ConnectionString))
                    {
                        return del(db);
                    }
                }
                catch (Exception ex)
                {
                    if (ex.InnerException is MySqlException &&
                        ex.Message == "Unable to connect to any of the specified MySQL hosts.")
                        if (_manager._dbConnection.GoToNextServer())
                            continue;

                    throw;
                }
            }

            return default(T);
        }
        # endregion
    }
}
This class acts as a layer between Entity Framework and my app, enabling me to easily do things like failover while still keeping the simplicity of EntityFramework. For each EF model I simply have to create a version of this class that uses the proper connection string and model. Example time:
 
// Standard entity framework call:
using (var db = new estatements_adminEntities())
{
    var usersList = db.users.Include(u => u.clientgroup).Include(u => u.client).ToList();
    // do stuff..
}

// The same call using AdminDb:
var usersList = AdminDb.RunQuery(db => db.users.Include(u => u.clientgroup).Include(u => u.client)).ToList();

Wednesday, November 23, 2011

Serializing dates in MVC (update)

As I was falling asleep in bed last night, a thought went through my mind: "Since I'm using this Regular Expression every time I load the page, isn't it rather wasteful to be compiling it every time?" So when I got up this morning, I looked up "Compiled Regular Expressions", and read through this page: http://www.dijksterhuis.org/regular-expressions-advanced/. Not only did I pick up some tips on compiling regular expressions, I also discovered a much simpler (and likely quite more efficient) way of replacing the MS-formatted date with my date string. In my previous method, I was casting the Regex matches to an IENumerable and using Linq to iterate through them; in the updated version, the iterative process is handled internally by the Regex object. All I have to do is tell it to use my FixDates method whenever it encounters a match. In the process of updating my code, I also added support for custom Date Formats and set the default to "yyyy-MM-dd".
private static readonly Regex DateRegex = new Regex(@"\\/Date\((?<ticks>\d+)?\)\\/", RegexOptions.Compiled);
/// <summary>
/// If true, date-specific serialization code will be run;
/// If false, date-specific serialization code will not be run.
/// </summary>
public bool HasDates { get; set; }
/// <summary>
/// The date format string to be applied. Defaults to "yyyy-MM-dd".
/// </summary>
public string DateFormatString { get; set; }

/// <summary>
/// Replaces MS-formatted dates with date strings
/// </summary>
/// <param name="match"></param>
/// <returns></returns>
public string FixDates(Match match)
{
    return new DateTime(1970, 1, 1)
        .AddMilliseconds(long.Parse(match.Groups["ticks"].Value))
        .ToString(DateFormatString);
}

/// <summary>
/// Serialize the object to JavaScript and
/// perform extra formatting on the serialized string as necessary
/// </summary>
/// <returns></returns>
public string Serialize()
{
    JavaScriptSerializer serializer = new JavaScriptSerializer();
    var serializedData = serializer.Serialize(Data);
    if (HasDates)
        serializedData = DateRegex.Replace(serializedData, FixDates);

    return serializedData;
}
Click here to download the updated code.

Monday, November 21, 2011

Serializing dates in MVC

Update: I've rewritten the code for simplicity and efficiency. You can check it out here.
Ah, dates. Simple little things, really, especially when working with .Net and JavaScript, right?
Well, sort of. See, Microsoft decided that when serializing DateTime objects, they should be formatted like this:
"\/Date(12345678)\/". Unfortunately, without running eval() on that, that format is useless.
I've never really had this issue before, because when working with dates on the client side, I always use strings.
So why the issue now?
Because, along with using MVC, I decided to use the Entity Framework, which made it really easy to return a bunch of objects from the database:
return Json(db.Tasks.ToList());
In the past, this would have been more like:

DataTable dt = new DataTable();
dt.Load(command.ExecuteReader());
return dt.Rows.Cast<DataRow>().Select(
                r =>
                new
                    {
                        id = (int) r["id"],
                        name = (string) r["name"],
                        date = ((DateTime) r["date"]).ToString("yyyy-MM-dd")
                    });

Now, I could simply go on using the select statement, but I just couldn't bear the thought of doing that everytime I had to work with dates (since that is all the time). So instead I wrote a custom Json serializer derived from JsonResult. I've uploaded the source code (link at bottom), but the real magic is in these lines:

public bool HasDates { get; set; }

public override void ExecuteResult(ControllerContext context)
{ 
    //... a bunch of code ...
    if (Data != null)
    {
        JavaScriptSerializer serializer = new JavaScriptSerializer();
        response.Write(HasDates ? FixDates(serializer.Serialize(Data)) : serializer.Serialize(Data));
    }
}   

public string FixDates(string data)
{
    var matches = Regex.Matches(data, @"\\/Date\((?<ticks>\d+)?\)\\/").Cast<Match>().ToArray();
    for (int i = matches.Length - 1; i >= 0; i--)
    {
        var match = matches[i];
        data = data.Remove(match.Index, match.Length)
            .Insert(match.Index,
                    new DateTime(1970, 1, 1).AddMilliseconds(long.Parse(match.Groups["ticks"].Value))
                        .ToString("yyyy-MM-dd HH:mm:ss"));
    }
    return data;
}

The code simply loops through the standard serialized result and replaces any MS-formatted date strings with my kind of date string (yyyy-MM-dd HH:mm:ss eg. "2011-11-21 19:48:13").

Updated return call:
return new ProperJsonResult() { HasDates = true, Data = db.Tasks.ToList() };

Click here to download ProperJsonResult.cs

Wednesday, October 26, 2011

Rectangle.ReallyContains()

So I just wrote an extension method, Rectangle.ReallyContains(Point p).
Seems the XNA folks and I disagree on what Contains() really means. For example, let's say a rectangle has a left value of 0, a top value of 0, a right value of 5, and a bottom value of 5.
To me, the corners of this rectangle are as follows: (0,0), (0,5), (5,5), (5,0).
So then, rectangle.Contains(new Point(0,5)) should be true.
According to the XNA folks, this is not correct. From what I can tell, the bottom and right values are the numbers directly after the rectangle ends (so the rectangle goes from 0 to - but not including - 5).
I'm sure their way makes sense in a lot of scenarios. It doesn't make sense for mine, and I just spent a good 10 minutes figuring that out.
*rant finished*

Nate's Roguelike Game

So I recently rediscovered ADOM (http://www.adom.de/) and after playing it for a while (and dying many times) I've decided to create my own roguelike game.
The game is located here: http://natesroguelikegame.codeplex.com/.
There are no releases as yet - so far, the only thing working is random room generation.

Friday, September 30, 2011

Morgan wrote a program!

Thanks to Microsoft SmallBasic, I coached Morgan through writing her very first program today!
It was a lot of fun to show her a bit of the development world and explain a little bit about OOP.

AStar in Java.. and this WYSIWYG editor sucks

I'm on another game programming kick. This happens every several months or so, I play around with it for a while before getting lost in the enormity of it all and wandering away again.
This semester, I'm taking a programming class which uses Java. Not being at all familiar with Java, I was pleasantly surprised to discover that it is very similar to my main language, C#.
While investigating Java game programming I came across jMonkeyEngine (a Java 3d game engine), and while going through some tutorials on their site I came across an AStar tutorial.
I decided to gain some Java experience by implementing AStar in Java. I got a basic version working pretty quickly, but it wasn't taking the fastest route - it was taking 9 blocks to do what should only have taken 8.
I kept playing around with it and got it down to finding the fastest route, but it wasn't nearly as efficient as it could be (it kept searching blocks that it didn't need to). I fixed this by recording what iteration the algorithm was on when each block was discovered, then weighting my search to sort first by distance (asc), then by iteration (desc).
So, given the following blocks:

BlockDistance From TargetIteration
1101
3102
2201

the old version would pick either block 1 or block 3 when searching; the new version would always pick block 3. Anyways, the end result is that the path my program finds is identical to the one in the tutorial:

Not really a big accomplishment, but it's exciting for me, so I figured I'd post it.


On a side note, I just changed the font to Arial then went back to the HTML and ugh! It took my nice simple text and added a span on every line with inline styles all over the place. Blechh.


Download Sample (Open the HTML page)
Download Source Code (Netbeans project)