Impressions from the keynote at Teched New Orleans 2013

It started with jazz...

...and continued with Brad Andersson driving an Aston Martin on stage.

Key areas

BYOD

Microsoft are bringing in new tools to handle a company's devices with more control over personalization, policies and separating company data and company apps from personal stuff. It seems easy to register your own device for workplace security using two factor identification and the new concept of work folders mimics the folder replication of dropbox in a enterprise fashion.

The concept of just cleaning out the computer by leaving the workplace was quite impressive.

Azure

Azure gets cheaper and a new billing model. Now you won't have to pay for stopped virtual machines and the billing is per minute. The MSDN Azure licence seems changed with a free azure usage amount of 50, 100, 150$ for professional, premium and ultimate subscriptions respectively. There have been improvements on keeping check on how much money/time you've used up.

Visual Studio 2013 / TFS 2013

More focus on ALM which will be on a higher level than in VS 2012, so that you can have a hierarchical view going from department to projects, subprojects, features, tasks etc...

A new HUD-feature in VS2013 adds information about references, tests and source code changes on a method by method basis. It looked really useful.

VS load testing now got cloud support so you can run your existing load test through the cloud.

Microsoft has acquired Inrelease which will bring a workflow-view to deployment and releases handling stuff like approvals and configurations.

Team rooms introduces a new collaboration space in TFS giving project members easier access to what is happening in their project.

Big data

Sql Server 2014 and new additions to Excel gives new possibilities to explore big data.

The very, very impressive geoflow in Excel gives thematic 4d-mapping with drilldown capabilities. See it as google earth with graphs and a possibility to animate time series. Looks amazing. There is also a new data explorer in Excel.

Ps. Swedes were sending most attendants to New Orleans of all countries in Europe. Well done!

Stranded in Chicago

I missed the ritual preflight-beer at Arlanda, maybe that is why I am stranded in Chicago (United blamed the tornadoes, but I know it was the missed beer...). 

Continuing to Raleigh tomorrow and New Orleans on Sunday.

But right now I am stuck with a complimentary night at an anonymous hotel in the middle of nowhere in Chicago.  Can't wait to move on.

Preparing for Teched

Just a week left before I head to Teched in New Orleans. It will be fun! ...and it will probably be hot as well.

I will try to put my impressions on the blog while I am there, so this programming blog will double as a trip report for the first week in June.

'Til then...

Over and out!

Tricky stuff with Sql Server Spatial part 2

So you have this Sql Server 2012 with spatial datatypes and you want to read it with C#.

The first thing to do is to add a reference in your project to Microsoft.SqlServer.Types where SqlGeography and the likes reside.

The next step would be to make a query and read the results using a SqlDataReader. Something like this:

                result.Location  = (SqlGeography)reader["Location"];

The problem is that this will result in a System.InvalidCastException with the message:

{"[A]Microsoft.SqlServer.Types.SqlGeography cannot be cast to [B]Microsoft.SqlServer.Types.SqlGeography. Type A originates from 'Microsoft.SqlServer.Types, Version=10.0.0.0, Culture=neutral, PublicKeyToken=89845dcd8080cc91' in the context 'Default' at location 'C:\\windows\\assembly\\GAC_MSIL\\Microsoft.SqlServer.Types\\10.0.0.0__89845dcd8080cc91\\Microsoft.SqlServer.Types.dll'. Type B originates from 'Microsoft.SqlServer.Types, Version=11.0.0.0, Culture=neutral, PublicKeyToken=89845dcd8080cc91' in the context 'Default' at location 'C:\\windows\\assembly\\GAC_MSIL\\Microsoft.SqlServer.Types\\11.0.0.0__89845dcd8080cc91\\Microsoft.SqlServer.Types.dll'."}

If you read the message you actually can understand what's going on. Even though you put a specific reference to the types in Sql Server 2012, it still seems as if C# is trying to read with the old version of Sql Server (2008).

The easiest way to fix this is through the connection string and the Type System Version construct that appeared in .Net 4.5.

My first attempt on a connection string was like this (and yeah, my computer is named GAAH):


    <add name="Main" connectionString="data source=GAAH\SQLEXPRESS;initial catalog=SpatialSample;integrated security=True" />

By adding Type System Version we can tell it that we will use the 2012 version of Sql Server Types.

    <add name="Main" connectionString="Type System Version=SQL Server 2012;data source=GAAH\SQLEXPRESS;initial catalog=SpatialSample;integrated security=True" />

Now youre just a read away from your spatial objects.

Tricky stuff with Sql Server Spatial part 1

Dear reader(s),

Last week I held an internal lecture about SQL Server Spatial and the basics of GIS here at knowit.
I really like talking about GIS as it is about the world and big amounts of data and I do like the world and big  amounts of data.

Well... where I am? who am I?

Oh, yeah. While preparing I noticed a few things that can be a bit tricky with SQL Server Spatial.

This is the first one.

Preparations:

I use a table called test which basically holds one geography column named spatial and another varchar() called name.

Inserts:


INSERT INTO Test (Name, Spatial)
VALUES(
    'Polygon',
    geography::STGeomFromText(
         'POLYGON((-1 -1, 8 -1 , 8 8 ,-1 8, -1 -1))',4326));

So this basically inserts a polygon into the world. As you can see it is square and crosses the equator. The number 4326 is called the SRID and basically means that the coordinates are calculated according to the projection WGS84. (projections basically means that you get different sets of coordinates depending on how you try to put the round globe on to a square piece of paper (a map). Look here for more info.).

Well, besides the 4326 this is no sweat... now try doing it this way:


INSERT INTO Test (Name, Spatial)
VALUES(
    'Polygon',
    geography::STGeomFromText(
         'POLYGON((-1 -1, -1 8 , 8 8 ,8 -1, -1 -1))',4326))

Exactly the same call, but we put the coordinates the opposite way. Now it is no longer a square polygon crossing the equator but instead it is all but a square polygon crossing the equator. It is covering the rest of the world.

Depending on if you add point clockwise or counter clockwise gives you two totally different polygons.

Why?

Well the world IS actually round meaning that any set of points that makes up a polygon on the surface of the world has two meanings. An interior version and an exterior version. Counter clockwise gives the interior version, clockwise the exterior. Easy to forget...

More about it here.

Taxonomy pt 3

In my last adventure in taxonomies I had hopes of using Neo4J and Neo4JClient to implement a taxonomy. However, I could never implement the Cypher-queries that worked in Neo4J through the .Net-library Neo4JClient. I got a "400 bad request"-error, struggled, tore my imaginary hair, gave up and changed the route to QuickGraph.

QuickGraph is a graph engine and not a database. It is not a big issue when it comes to a small taxonomy, but would pose problems for huge taxonomies since it holds all data in memory. For me it was a relief, since QuickGraph is something I know and feel comfortable with.

Installation of QuickGraph is straightforward. Just download, build and set a reference.

As a reminder a taxonomy is something where the meaning of a certain term is valid over a specific time. In this case the time is in which revision the term was defined. Like this:

I am using a directed graph where each revision of the taxonomy is descendant of the last revision.

As a practice set I used the gulls in the last blog (check it out).

Step 1.  Create an object that will hold each gull.


    public class Taxon 
    {
        public Guid TaxonId { get; set; }
        public string ScientificName { get; set; }
        public string CommonName { get; set; }
        public int Revision { get; set; }
    }


Since each scientific name can be used in several revisions I use a guid to know which unique taxon it is.

Step 2. Instantiate a directed graph.

BidirectionalGraph<Taxon, IEdge<Taxon>> graph = new BidirectionalGraph<Taxon, IEdge<Taxon>>();

This basically means that the nodes will be of the type Taxon and that the edges will connect two Taxon objects. Bidirectional means that each nod will no who it is descendent from and who derives from it.

Step 3. Implement the InsertTaxon function.


        public void InsertTaxon(Taxon taxon, List<Taxon> ancestors, BidirectionalGraph<Taxon, IEdge<Taxon>> graph)
        {
            graph.AddVertex(taxon);
            foreach (Taxon item in ancestors)
            {
                TaggedEdge<Taxon, double> outEdge = new TaggedEdge<Taxon, double>(taxon, item, 1);
                graph.AddEdge(outEdge);

            }
        }

We're basically inserting the taxon for a new revision of a taxonomy and connects it to all taxons it derives from. By using a TaggedEdge we have a weighted graph. I just default it to one though.

Step 4. Implement the FindTaxon method.


        public IEnumerable<Taxon> FindTaxon(String scientificName, int fromRevision, int toRevision)
        {
            //Find the taxon with the same scientific name from the chosen revision
            Taxon searchTaxon = graph.Vertices.Where(s => s.ScientificName == scientificName && s.Revision == fromRevision).First();
            //use that taxon to find the possible taxons at the to revision
            List<Taxon> results = new List<Taxon>();
            results.AddRange(GetAllDescendantsAtTime(searchTaxon, toRevision));
            return results.Distinct();
        }


        private List<Taxon> GetAllDescendantsAtTime(Taxon searchTaxon, int toRevision)
        {
            List<Taxon> results = new List<Taxon>();
            foreach (Taxon inNode in graph.InEdges(searchTaxon).Where(s => s.Source.Revision <= toRevision && s.Source.Revision > searchTaxon.Revision).Select(n => n.Source))
            {
                if (inNode.Revision != toRevision)
                    results.AddRange(GetAllDescendantsAtTime(inNode, toRevision));
                else
                    results.Add(inNode);
            }
            return results;
        }

So what we basically do is to find what say a Herring Gull was at revision 1 of the taxonomy and then see what it might be at revision 3.
We accomplish this by walking the graph recursively until we find all revision 3 species that are  based on the Herring Gull. The answer would be European Herring Gull, Yellow Legged Gull, Vega Gull, Caspian Gull and American Herring Gull. 

To note: This approach would need a specific graph for each taxonomy and, yeah, there are probably some algorithms that solves the problem faster than mine.

For me: This is perfect! Now I will just make a complete interface to it and use it in all my birding projects!

See you!

Taxonomy pt 2

In my last entry I discussed the special constraints and problems that occur when you need to implement a classification system that changes over time.

This time we will take the discussion a little bit deeper and look at the basic API of a taxonomy component.

A taxonomy is a graph where each taxon has a limited life span and is traceable through previous revisions of the taxonomy.
If I would use birds as an example (and I really like to do that). The Armenian gull, Larus armenicus, is considered a specie by The Association of European Rarities Committees.
It was first considered a sub specie of Herring Gull (L. argentatus), but after that specie was split into European Herring Gull, Larus argentatus, American Herring Gull, Larus smithsonianus, Caspian Gull, Larus cachinnans, Yellow-legged Gull, Larus michahellis, Vega Gull, Larus vegae and the Armenian Gull, Larus armenicus.

To complicate stuff further another taxonomy, namely birdlife.org, doesn't consider it to be a valid specie but lumps it together with Yellow-legged Gull (Larus michahellis).

So... depending on when you see this gull and which taxonomy you use it can be either a Herring gull, an Armenian gull or a Yellow-legged gull and then you're only checking two taxonomies and believe me, there are more out there...

What does this tell us?

1. A taxon has a time span.
2. A taxon is derived from one or more taxons.
3. A taxon is dependent of its taxonomy and several parallel taxonomies may exist.
4. To find a taxon from its key (the latin name in this case), you will need to know the key, the time and the taxonomy.

To find out that the Armenian gull nowadays is considered a sub specie of Yellow-legged gull in Birdlife, I would have to backtrack the taxonomy graph to find the Armenian gull and then follow it to present time to see that it has been included in Yellow-legged gull. 

Even though this example is about birds, the same will apply more or less to any other type of taxonomy.

In pseudo-code the core functions would be:

• Taxon InsertTaxon(Taxon taxon, List<Taxon> ancestors, DateTime validFrom) to insert a taxon based on zero or more ancestors.
• Taxon FindTaxon(String key, DateTime when, Taxonomy string) to find a taxon given a key, time and a certain taxonomy.

The signature can of course differ, but the basic design will be the same.

Next time we will take a look on how we can implement this.

Til then... Bye, bye!