Version Control

I’ve been thinking lately about the adoption of version control. One of the common fears of using version control is that its permanent. All mistakes are visible and are in fact, highlight. Originals are updated with new commits which detail only the change and a nice comment. There is no hiding mistakes. The thing is

Mistakes are what make us human.

Everyone makes mistakes, we need to accept that fact. In doing so we can move forward faster.

Keyboard mapping [SQL 2014]

I’ve been working on SQL Server 2014 lately and one of the annoying problems I’ve stumbled across using SSMS is that F5 no longer means execute the SQL query, it now means Debug. Which brings up an annoying popup (least for me) about configuring my Windows Firewall.

This can be changed in SSMS by going Tools > Customize… > Keyboard…

The default is:

Debug.Start (F5 (Global))

And the traditional is:

View.Refresh

You can also do it by setting it to Visual Studio 2010 Compatible

4th Anniversary of contributing to Rosetta@Home

Tomorrow, Sunday the 7th December 2014, I will be celebrating 4 years of contributing wasted CPU cycles to the excellent Rosetta@Home project.

rosetta_at_home_logo

Rosetta@home project is determining the 3-dimensional shapes of proteins through research, that may ultimately lead to finding cures for some major human diseases such as HIV, Malaria, Cancer, and Alzheimer’s. The project is lead by Dr David Baker at the Univerity of Washington.

You can find out more about the Rosetta@Home project on their website – http://boinc.bakerlab.org/.

Not only will this be my 4th year contributing, I have also surpased the

1,000,000

certificat

credits mark. I think in part this is because of my new i5 CPU I got for my birthday this year.

Bimw0KOCIAA0mmX

 

Technical Analysis on Schools websites in England

So in my last blog post I asked, are you ready for IPv6? The post came about when I was looking at Schools MIS data, which Graham, Joshua and myself have being look at to see who are the big movers and shakers in the Schools administration software (MIS) arena. Data is collected by what software suppliers a school uses to submit the School Census (in England) which is requested under the Freedom of Information (FOI) from Department of Education(DfE) (saving having to FOI every individual school). In order to enhance this data I was joining the data onto the general schools data that can be extract from EduBase. Looking at the data I notice that the website addresses listed in the extract was of extremely poor quality. A number even had email addresses listed!

This got me thinking, are schools ready for IPv6? If Sky are running out of IPv4 addresses and offer IPv6 only connections at a lower price, how many parents are going to jump on the deal only to find out they can’t access their child’s school website later on. After a bit of scripting and a support call to Mythic-Beasts to enable a response in JSON that I could automate, I had the results. It wasn’t good.

Still, no-ones ready for IPv6 are they, sure they’ll be ready in time. Won’t they?

We can only judge the future from what we have suffered in the past

Themistocles , 300: Rise of an Empire

To this effect, I’ve gathered data to look at:

  • šDomain registration correctness
  • šContent management systems
  • šDocument type definition
  • šRaw HTML homepage size
  • šGoogle Analytics
  • šIPv6 readiness

At the moment I’m still creating the presentation detailing my findings, but you can download the raw data from: https://github.com/matt40k/SchoolsWebsites-England

Are you ready for IPv6?

For a long while now it has been known that IPv4 will run out of available publicly assignable IP addresses. IPv4 address are 32 bits wide and quite simply can’t cope with the demand of the modern world – despite NAT’ing. IPv6 looks to resolve this by have not only more digits, but also using hexadecimal.

For example, an IPv4 address looks like this

192 . 168 . 0 . 1

4 parts each running from 0 – 255.

Now lets look at Facebook IPv6 record:

2a03 : 2880 : 2110 : df07 : face : b00c : 0 : 1

Now that’s 8 blocks, each one being 4 hexadecimals – so that’s 0 – 9 then a – f (so 16), so that’s 16 x 16 x 16 x 16 – so 65,536 in a block vs IPv4 255, and there are 8 blocks… that’s 128 bits, starting to see how massive the IPv6 range is?

Because of the massive IP range, you in effect get to bypass the IPv4 tax – in fact Mythic-Beasts has already started offering IPv6 only servers without the IPv4 tax. That’s not the only thing, NAT becomes obsolete which means gaming on the XBOX One becomes faster as you don’t have to setup port forwarding and put extra load on your router – also new Microsoft devices (Windows PCs and XBOX One) have a preference for IPv6 over IPv4, this could be because IPv6 has cleaner routing? IPv6 has been designed for the future, security has been account for and isn’t just tacked on.

When is IPv6 coming out? Well, it’s actually out, and it’s been out for a long while. RIPE, who are responsible for issuing IP addresses (both IPv4 and IPv6) in Europe has been doing it since 1999 and World IPv6 Day was back in 2011.

So, are you IPv6 ready?

You can check if your ISP has setup IPv6 so you can access those IPv6 servers from your computer by going to: http://test-ipv6.com/

And you can check if your website is IPv6 ready by going to: https://www.mythic-beasts.com/ipv6/health-check

I’m guessing the answer is no.

Thankfully, you’re not alone, but it’s something to think about, especially as all the major ISP in the UK are gearing up – http://blog.mythic-beasts.com/2014/10/22/ipv6-support-in-the-uk/ and http://www.ipv6.org.uk/2014/11/20/ipv6-council-meeting-oct2014/

Slow progress

I’m currently getting bogged down with other projects and unfortunately my final sprint of SIMS Bulk Import has grind to a halt. On the plus side I managed to swash a few more bugs last month, get a code signing certificate. This will mean all future releases will be signed as coming from me which is great news, it adds a layer of confidence that the code you run is unaltered by a third party. I’ve also started reviewing my other projects, ensure that the code on the public repository is up-to-date, I’ve also switched over to Git and copied them all over to GitHub. At bit more on that later.

Part of the updated process was to use a Continuous Integration server – specifically TeamCity. Simply put, these allowed the builds to occur on a dedicated box which made the whole process quicker and easier – it also added confidence as it ensured that everything required was commit in the repository. The only downside to this is the cost – although renting a VPS from OVH is cheap, it still isn’t cost effective due to the limited amount of time I actually use it, I’m therefore looking at moving it to Azure as you only pay for provisioned resources and your can de-provision servers and only pay for the storage. Alas this means more messing about setting up servers.

TeamCity AssemblyInfo Patcher not working

Tonight I was trying to get TeamCity to auto-update the version number for my SIMS Bulk Import application, however the simple AssemblyInfo Patcher just failed to work. No errors. Nothing.

Then I read the manual, again…

Note that this feature will work only for “standard” projects, i.e. created by means of the Visual Studio wizard, so that all the AssemblyInfo files and content have a standard location.

So I created a new package which placed the AssemblyInfo.cs into the Properties folder, mine was in the main directory, moved it into the subfolder and bang, it works. Awesome.

 

WIX installer why you no MSBUILD no more?

Last night I chopped up my WIX installer project – I basically split the actual installer parts into one file – CoreMsi.wxs, the versions into a variable file – Version.wxi and the file list into a separate fragment file – SimsBulkImport.wxs. This worked.

Great I thought, simple PowerShell script to generate the file list, I can even grab the main repository complied bin as a artifact dependency – got to love TeamCity.

Problem was, it fails to build.

error CNDL0104: Not a valid source file; detail: ‘.’, hexadecimal value 0x00, is an invalid character. Line 2, position 1.

bit of trial and error later I figured it out. It was my old friend Mr Encoding. After adding

-Encoding “UTF8″ 

to the end of the Out-File and it builds again.

SIMS Bulk Import

Overview

If your in the UK, the chances are your local school is using SIMS .net. Just checkout the stats on their site. 22,000 schools taking 2,500,000 children’s attendance, every day. That’s impressive.

So what is SIMS .net? Well its a MIS system, but what is a MIS system? In the simplest terms its a database that holds the school records, for both students and staff.  So it makes sense that you are going to want to interface with it as it’s your at your core when it comes to data sources.

Now extracting data is pretty straightforward. Capita have created a custom reporting engine that allows simple report creation that can then be scheduled and produce exports of data. The problem is getting data back into SIMS .net. Take for example identity management, it’s easy enough to export a list of students, then write a PowerShell script to generate user accounts, but wouldn’t it be good to get that username added back into your core data source? Or what about adding in new students home email addresses and telephone numbers? Well it isn’t that straightforward to bulk import. Although Capita provide a API it isn’t as straightforward and certainly isn’t as friendly as a RESTful web service and certainly requires a programming background to understand it which rules out many schools being able to use it.

Back in May 2012 I was working on a SIMS support desk doing technical support and one of our customers asked if he could bulk import email addresses back into SIMS .net. This resulted in me asking Capita for documentation regarding the API which they provided, at no cost, along with a few snippets of example code. I then spent the following nights coding away at home to what has become SIMS Bulk Import.

About

So what are SIMS Bulk Import features

  • It’s free – Thank you Phil Neal for waiving the licensing costs :)
  • It uses the SIMS .net Business Objects (no large Capita charges for corrupting your SIMS .net database!)
  • Imports from CSV, Excel spreadsheets and XML files
  • Matches the file fields with SIMS .net fields

The future

I’ve since moved away from SIMS support and I’ve been trying to find a suitable new home for SIMS Bulk Import. Whether that’ll be Capita writing a replacement, or someone else taking up the project it isn’t clear. For now I’ll continue to support it. But to as part of this I’ve moved the source over to GitHub, the releases will still be via CodePlex.  I’ve also created a separate organisation on GitHubSIMSBulkImport, which effectively “owns” the code. I fork the code from  SIMSBulkImport into my personal repository, and do a pull request to merge the code back into the main SIMSBulkImport repository. This will help should the “owner” of SIMS Bulk Import change in the future.

I’ve also created a number of sub-projects – the SIMS interface library will be moved from Unfuddle onto GitHub, also the installer element will become a separate repository. I’m looking at setting up TeamCity to automate the build process so building and creating releases is a lot more streamlined and less time consuming. Also a simple web site setup giving easy advise for getting started.

Another change is a web API – this allows checking for newer versions and secure uploading of log files to the developers (OK, me). Mythic-Beasts have kindly donated a server hosted in Cambridge, UK to host this. This written in PHP using the Laravel framework, I’ve started writing it based on some of Bobby Allen original code.

Going forward I would like some (official) recommendation from schools and support teams, ideally I’d like to put together some sort of list or program detailing where you can get support from, for example:

Your Local SIMS Team supports SIMS Bulk Import!

Your Local SIMS Team is a SIMS Bulk Import silver partner!

It’s just difficult making a business case to people when your product is free. The main reason I want to do this is I’m find the odd problem where someone need to remote in, this is difficult without some sort of support agreement and T and Cs – most of the problems are actual SIMS problems like the SIMS client hasn’t be installed correctly which their SIMS support team could resolve. I appreciate working in the dark isn’t easy and you can’t know about every SIMS third party product, so your going to get a element of from pillar to post. Ideally I need a main backer to act as data controller with regards to the log files, at the moment I’m investigating if I could get around it by limiting the data it could spit out but that’s going to limit the usefulness of the logs. It’ll also been good to get the program digitally signed.

If anyone is willing to help out feel free to drop me a email – matt [at] matt40k [dot] uk