What have you done for me lately?

Don’t worry! This is not yet another “I haven’t blogged because…” post. It’s similar though: it’s to aggregate all the cool material I’ve been producing lately instead of writing on my personal blog!

Without further ado, here’s the goodies from the past:

Oh yeah, there’s also some stuff going on in the present:

Finally, here’s some cool stuff I might possibly do in the near future:

  • I’ve worked several months in a Java/Maven/Spring project, which lures me to write about the differences between Java and C#. No promises though!
  • 2018-01-xx: excited to start on a project that uses ClojureScript, hurray for opportunities to learn stuff! Might even write about it, here or on Infi’s blog.
  • 2018-xx-xx: blog post about Top 100 Games is in progress, though no promises here!
  • 2018-xx-xx: a side project that gets sidelined a lot, I still intend to finish EmpGrid some time.

Several things on these lists I want to blog about, but I’m not ready to commit to anything yet. Stay tuned!

Learning to Linux: the Road to Becoming a Power User

Let’s first set the scene: I’m participating in an Infi study group and would have to present to colleagues about “WTF is Linux?”. Interestingly, I am a total Linux beginner, and they were ranging from intermediate to advanced skill levels. Joy!

So, what’s on the to-do list? Three things:

  1. Understand history and context of Linux;
  2. Get to know what you don’t know yet;
  3. Learn what the tools to learn more are;

Basically, I’m looking for all the things I need to get started and to keep going. Note that all this is aimed at folks that are at least to some degree a power user.

History and Context

First there was Unix, which was great, but it was closed source and not very portable. So Richard Stallman set to create a clone: GNU. However, the kernel “GNU Hurd” didn’t work out so well at the time. So Linus Torvalds wrote the Linux kernel in the 1990s, and together they now form the Operating System “GNU/Linux”, often referred to as just “Linux”. History has by now shown that GNU/Linux was far more popular than GNU/Hurd.

Before we dive into Linux itself, let’s consider its place amongst other Unix-like OSes. There are both open source ones such as Minix and FreeBSD, and closed source ones like Solaris. All of them are fully or partially compatible with “POSIX“: a standard for OSes. You can get the full picture at Wikipedia, but here’s a simpler version:

Unix-like OSes overview

Zoom in on the green “GNU/Linux” box, there turn out to be very many Distros. A distro being the Linux kernel, GNU tools, a package management system, and possibly a window system and desktop manager. Again, there’s a full picture at Wikipedia, but I made a more condensed overview:

Linux Distros Overview

The top ten distros for 2016 according to DistroWatch are as follows:

Rank Distro Package Manager Default Desktop(s) Comments
1. Mint dpkg Cinnamon, MATE, Xfce, KDE community-driven spinoff from Ubuntu with a lot of out-of-the-box multimedia stuff
2. Debian dpkg Gnome reliable Linux distro that can be used plain, but is also the ancestor for many other distributions
3. Ubuntu dpkg Unity (moving to Gnome 3) beginner-friendly spinoff from Debian, both for desktop and server environments
4. openSUSE rpm None, Gnome, KDE community program sponsored by SUSE, to promote use of Linux everywhere
5. Manjaro pacman XFCE, KDE, Gnome power and flexibility of Arch, but with a friendly installer and some more GUI stuff added
6. Fedora rpm Gnome latest and greatest features that will also go into Red Hat Enterprise Linux
7. Zorin dpkg Gnome 3 spinoff from Ubuntu for newcomers accustomed to the Windows GUI
8. elementary dpkg Pantheon spinoff from Ubuntu with the look and feel from Mac OSX with a few custom apps
9. CentOS rpm None, Gnome, KDE basically Red Hat Enterprise Linux without the enterprise license and support fees
10. Arch pacman CLI very minimalistic Linux distro, requiring the user to take control of the OS

There is also a pretty good Youtube video that gives some context about this same list.

Oh, and obviously I also have to do an honorable mention for the weirder, more “clickbaity” Linux distros: Satanic Ubuntu, Christian Ubuntu, Damn Vulnerable Linux, LinuxBBQ, Slackintosh, and Stresslinux. For the most part I have no clue what they do (if anything at all), I’ll leave that as an exercise to the reader.

What to Know

First thing to do when getting started with a new technology is finding out how much you don’t know yet. Thinking about the Dunning-Kruger effect…

Dunning-Kruger effect

… I would like to start at the Valley of Despair right away. And I’d like a clear path of that slope ahead of me. Please.

Now obviously you can’t start from zero; even reaching the Valley requires some knowledge. So investing some generic startup time seems good. Here’s a few suggestions:

But where to go from there? What is there to know? Well, there’s a few obvious resources that serve as very natural check lists:

In addition, it seemed helpful to try to do one or two nontrivial in-depth tasks in a VM, and see what you encounter. For example setting up a LAMP stack to host WordPress in a VM, or cloning a cool project and getting all dependencies (like Java, ugh!) up and running.

At the end of my research, and with help of my study-buddies, I conclude these are the few essential skills that I’m missing to be productive with Linux as a developer or sysadmin:

  1. Command Line. Without solid knowledge here you’ll feel lost for any nontrivial task in Linux. Some “introduction to Linux” articles tell you it’s not really needed (depending on the distro you choose), but I disagree, and think you’d get frustrated unless you learn to use it.
  2. SSH. Related to the Command Line, knowing how secure shell works is essential to managing a remote Linux instance (e.g. a VM in the cloud). This includes knowledge on managing SSH keys.
  3. Nano or Vim or similar. Also related to the Command Line. Often you’ll have to edit a config file from the command line, and being able to do that confidently is quite helpful.
  4. File System Hierarchy and Permissions. Knowing where to find stuff (and who has access to it) is essential to be productive. On a specific note, it seems important to know where things will be logged (e.g. where does a package install or server software keep its logs).
  5. Package Management and Installing Software (and troubleshooting that when it goes awry). Most interesting tasks on Linux require installation of libraries and software, and quite often this can go awry. You need to learn how to debug this.

My advice, at least to myself, is to sit down and carefully master these first. At least to some degree. Only then will other tasks on Linux (e.g. managing an NGINX server) not be a frustrating job anymore.

Learn to Learn

So how do you most efficiently climb this learning curve? Discoverability of essential info on Linux can be a bit of a problem. But here’s a shortlist we came up with:

  • man can be used with any other command as an argument, and it will explain all about it
  • info is GNU’s answer to man, intended to be more fully-featured (hyperlinks and such) and comprehensive
  • The “bropages” is like man but more succinct and example-based
  • locate, find, and which are commands for finding stuff on the file system
  • aptitude in Debian distros, or a similar tool for your set of distros, to help understand the package system
  • unix.stackexchange.com can be a great generic resource for Linux (and other *nix OSes)
  • https://askubuntu.com/ is quite a vibrant community should you choose to use that distro
  • LPIC (or similar) exam objectives can be a great study guide
  • read shell scripts before executing to learn
  • and as a final trick, you could do git init in a certain folder (e.g. /var/log) and use diffs to check how things change following your actions

However, perhaps unsurprisingly, the most common piece of advice when asking experts “How to learn?” is…. practice. Practice, a lot!


PS. This was cross-posted to Infi’s blog.

Beginnersplaining Linux to Experts

Working at Infi I’m involved in several study groups, one of which is about Linux. When we started the group it turned out we had six people and six different levels of experience. And I was the absolute newbie of the group.

We decided to set up a program divided in weeks, with one subject for each week, and a “Lead” for each subject. The Lead would research a lot about his or her main topic, and ask the rest of the group to merely do some light recommended reading. This is to get everyone at a base intro level for each topic. Every week we’ll have a session where the Lead would talk about his or her research, along with some open discussion.

Obviously, the first topic would be a “Linux Basics” session. It also seemed logical that the newbie (me!) would pick up this topic, because I’d have most to gain from researching the hell out of it.

Seems legit, right!? Except I now realize this means I’ll be presenting on Linux Basics to five colleagues that know a lot more about this topic than I do. Oh, and because it’s the first week’s topic I have only a week to prepare. Whelp!

I’ll just have to focus on learning a lot from all this. No progress without failure, and no progress without taking risk, I guess.


Foot Note: The Programme

We composed the Study Plan based on whatever the group members individually found interesting enough to research. This means it’s a bit of a weird assembly, but interesting nonetheless:

  • Week 1: WTF is Linux?
  • Week 2: Command Line
  • Week 3: File Systems
  • Week 4: Screen
  • Week 5: Web Servers (Apache & NGINX)
  • Week 6: Ansible
  • Week 7: .NET Core
  • Week 8: Logging & Smart Monitoring
  • Week 9: KVM, Virtualization, Containers

Entity Framework: Cascading Delete of Optional Related Entity

After several years of NHibernate, and a couple of years Dapper and NoSQL, I’m now working on a project that uses Entity Framework as its ORM. It’s a mature ORM at this point. However, it does give me a headache as I’m struggling to find ways to do certain things. I’ll admit that I’m trying to dive in without sitting down and doing a few hours of learning first; for sure I’ll be on Pluralsight some time soon.

Fair warning: what comes next is a highly specific, rather technical, and possibly stupidly uninformed dump of a problem I ran into. Feel free to skip this one: no hard feelings, and I’ll see you on my next post!

The main problem: I’ve got a repro, but it’s extremely similar to various other questions on Stack Overflow. Except… that my code contains the accepted (and often highly upvoted) answers’ code as well, but still doesn’t work as advertised!

What I’m trying to do is make sure that EF will delete a “Child” property (i.e. the database row) automatically when its “Parent” is explicitly deleted via the DbContext. Note that the Child is optional in my scenario.

Here’s the repro. Create a new class library and install EntityFramework (I used 6.1.3) and NUnit (I used 3.6.1). Then drop in one namespace these entities:

And then this DbContext:

And finally this TestFixture:

Make sure the “TestDb” database is available on your “(LocalDb)\cascades” instance (or use another Sql Server instance and database). Then run the tests, and you’ll get:

Test Failed – Delete_will_cascade.
Message: “Child”.
Expected: 0, but was: 1.

All I want is that a “Child” is deleted with its “Parent” once that is deleted. I know I can use a Sql CASCADE, and I know I could manually remove the Child from the context, but I want EF to handle this automatically, damnit!

For sure I’ve missed the obvious solution. I was somewhat hoping it would come to me while writing this. But perhaps I just need a good night’s rest.


References:

Job Hunting Checklist

Having recently switched jobs, now is probably the best possible time for me to write about the applicant’s side of vacancies and job searches in IT. Specifically, I want to give my take on maximizing the chance you’ll land a job at a place you’d want to work at. Put differently, below is (mostly) my own plan for finding a new job.

But wait: that’s not specific enough for a blog post yet! There’s a lot to job hunting. So let me narrow it down to the asynchronous ways to communicate with prospective employers.

What I mean here by “asynchronous“? Well, that includes just about everything that’s not real-time (face-to-face or phone conversations, chat, etc.). I will run down these “channels” one by one, including specific tips for each.

By the way, I hear you thinking: “But that sounds like just a big check list!?”. It is. But hey: check lists are great! Use this one to your advantage. Let’s get started with the basics.

General tips

Several things are important for all of the async channels. Remember:

  • Choose a great avatar you’ll be happy with for a long time to come, and stick with it. Everywhere! This will take a few years to settle in, but after a while it’ll help people recognize you across the board.
  • Be smart about your e-mail addresses. Preferably, you’d get your own domain, use addresses like some-org@yourname.com and github@yourname.com. Alternatively, use the + sign lifehack to differentiate. This will help you separate incoming mail reliably, lets you know who’s leaked or sold your e-mail, and gives you a minor edge with tech orgs because they’ll notice you’ve given things some thought.
  • Do or do not, there is no try. To put it in my own words: either do something right, or don’t do it at all. For example: it’s better to have no LinkedIn profile than an out-of-date one.
  • Try to do as many of the list below as possible, but consider this tip strictly lower priority than the previous tip!
  • Use language correctly, or at least as correct as possible. It’s easy to use a spell checker, proofread, do some proofreading, and do some more proofreading. Tip: read your writing (semi) aloud to yourself to find bugs in your writings.

Okay, so enough with the generic mumbo jumbo. Off to specifics!

Your CV

A lot has already been written about this, so I won’t repeat the basics. But I will make a few important additional points:

  • Decide: short or long. Both are fine. If you go short: try to fit everything on one single page. This is really hard (I could never do it!), but if you can manage it’ll demand respect and it’ll be useful to the reader. If you go long: use “scannable” text and styles (tables, bullet lists, emphasis, and so forth).
  • Use a (non-default) theme. The theme (and sticking strictly to it) will help prevent your CV from becoming ugly. Only try to make a “beautiful” CV if you know how to design or have help in this department.
  • Read “The Non-Designer’s Design Book. It’s a “Must-Have” if you insist on tweaking styles (like me), and when you’re also (like me) not a great designer. It’ll at the least help you prevent your CV from becoming ugly.
  • Use serif fonts. Sans-serif might look nicer on (non-retina) screens, but most of the time folks will read a print of your CV. On paper, a serif font is easier to read.

Open Source platforms (Github, Gitlab, etc)

This one’s not for everyone. Whatever you do or don’t do: be conscious about your choices. That doesn’t mean everything has to be perfect, but do mind the things that are easy to get right. Think about cleaning up utter garbage repos, using correct language when commenting on issues, etc.

If you do decide to work on open source, note that you don’t necissarily need to become a maintainer of a high esteemed project to be noticable. Here are some easier ways to get started:

  • Pet projects are something to put online. Place a big disclaimer that the code should be viewed as such, and you’re fine putting out non-production-worthy code.
  • Create a demo project that shows off how you prefer to code. It’s fine to “reinvent the wheel” or even create yet another “To Do App”, as long as you focus on the code being in “your” style. Place a big disclaimer that the code should be viewed as such, and you’re fine.
  • Create Issues for other projects. Creating an issue with a good repro is usually well-received, and displays qualities very important in day-to-day tech work.
  • Find up-for-grabs issues in a project you’d like to contribute to. Or even open your own (most projects will welcome improvements to documentation, for example).

On a side note, it can also be good to spend some time looking at the source code of open “famous” source projects in your tech stack. It hones your code-reading skills, and demystifies the black boxes they tend to be. Although this isn’t directly visible to prospective employers, it does have intrinsic value and might be a gateway to your own contributions some day.

Community Forums and Q&A (Stack Overflow, etc)

On the one hand I see this different from the previous point (Open Source), on the other hand it kind of isn’t. Specifically Stack Overflow (of which I’m somewhat a fan, as they distribute all content freely, for one because the data is licensed by users at least under CC-By-SA.

Asking questions is probably the easiest way to get started. Note that exemplary questions are a great way to show off your skill: being able to grasp and neatly summarize a reproducible problem is perhaps the number one skill a developer can have. I’ve asked multiple questions I’d proudly show off as part of my resumé.

Answering questions is a lot harder. This is another whole topic on itself so let me just sprinkle some basic advice. First, the “gamefication” can be fun but should not be your primary motivation. Instead, answer to learn about topics. Choose a low(ish) traffic tag to answer questions in, or you’ll get frustrated by the fastest gun in the west: other people will answer extremely fast so you don’t seem to have a chance. And finally: keep at it.

My first few questions and answers on Stack Overflow were pretty bad, but practice makes perfect! For every ten answers with even one upvote there’s at least one that turns out to help a lot of other folks. Seeing a small “+1” trickle in on an old question is a great, warm feeling.

A personal website

Get a personal domain, preferably something with your name in it. Make it a single page with at least:

  • A very basic bio: one or two lines of text about who you are.
  • A generic e-mail address (one that doesn’t matter if it gets a bit of spam).
  • Links to any social media accounts, e.g. Twitter, LinkedIn, but also Meetup, Goodreads, or other media that are professionally relevant.
  • Links to open source profiles, e.g. your GitHub profile or Stack Overflow flair.

Then, optionally, you could do any of these:

  • Link to your blog (see next section).
  • List of your projects, e.g. pet projects, portfolio stuff, your master’s thesis, etc.
  • Link to interesting other sites, e.g. tech blogs you enjoy.
  • A short occupation history if you’re inclined to make that public.
  • Some schema.org info and meta tags for SEO and generally showing off mad skills with details.
  • Ninja-counter-recruitment in your source code, just as organizations do.

A personal blog

Also not for everyone, but it certainly does help showing off enthusiasm, writing skills, and really whatever you want to show off. Blogging about tech is great, but mixing with non-technical posts works even better.

And in addition to exposure, there’s a far more compelling reason to blog: writing helps you organize your thoughts. Having to clearly express your thoughts or opinion about something should also make you think really hard about that thing.

Gaining a bigger audience is a challenge on its own, not usally directly related to the writing itself. So think long and hard before you decide to go down that road, because you’ll have to double down on community-building to get somewhere. But then again: I feel you can get a lot out of a blog without a big audience, too, by just writing to organize your thoughts.

Social Media

Warning: only do this if you enjoy it! The days of “needing” to be on social media to fit in are gone with Gangnam Style and the rest of 2016. Having said that, social media can be helpful in building your online profile.

First up: be consistent. Be consistent with your tag line, title, etc. Be conscious about what you make public and what you keep private. Don’t be afraid to create only a simple account that just links through to another platform; for example, you could create a LinkedIn profile that just tells folks to check your own website.

For the rest this comes down to personal preference. For me, Twitter and a wee bit of LinkedIn are primary professionally used social media. Google+ I find useless for most things, and Facebook is purely a thing for friends and family for me.

As a footnote it’s also worth noting that there are outliers. Goodreads is something I only use because I find the platform interesting, and I keep my professional bookshelf there. Also, Meetup.com has been popular the last two years, so I guess that also counts as a medium to be conscious about.

In closing

Bear with me while I digress a bit. Suppose you want to set up a perfect romantic date with someone. To maximize chances of success, you prepare all the things that can be reasonably prepared. You make sure there’s food, drinks, music, but also toilet paper, running water, etc.

Now let’s zoom in on the food. If you’re a chef or if you like cooking you might go out of your way a bit to cook something fancy, though you probably shouldn’t go too risky (unless that’s part of your charm…). If you hate cooking dinner you might order in something nice, and focus more on other things. In other words:

  • Do or don’t do! There is no “try”.
  • Min-Max All The Things™ that you do.

Back to the point. Maximizing your chances for a perfect job works the same as setting up that perfect date night. Be conscious about what you focus on. Be smart about what you do or don’t do. And whatever you decide you will do: do it right!


Footnote: his was also cross-posted to the Infi company blog.

New Beginnings

In less than two weeks, I’ll be starting a fresh challenge at Infi. I hope (and think) I will bring things to the table, but I’ve also already noticed that there will be a lot of new things for me to learn. I honestly feel that a large portion of your skills and knowledge can be best gained from other people (online, or in real life, doesn’t matter), which is why I’m very excited about this new beginning!

I’m pretty sure that this fresh start will influence what I’ll be writing about on my blog. I’ve got some ideas, but even more so I think additional concrete ideas will start to flow in the coming months.

But first, before we get to that, there will be (a) a very short vacation, and (b) a short period of silence while I get settled. After that, you can expect the reports to flow in again here. See you then!

A New Hope

Did I mention before that I write here to collect my thoughts and as writing practice? Well, in any case, after my previous monster post, and given that it’s the start of a new year, I figured I’d do a “New Beginnings” post. Then the sci-fi addict in me awoke and typed “A New Hope“, even though I’m a Trekkie at heart. But I digress.

I wanted to summarize which technologies and experiments I’m most interested in pursuing next. Let me try to list them in order of current preference.

1. Elasticsearch …. and Nest?

Elastic as a (primary?) data store intrigues me. It feels like a second chance for Lucene towards me (experimenting with Solr didn’t “click” for me). There recently was a major version released so it seems like a great time to start investigating. I’m hesitant about Nest though, which you’d kind-of have to use if you want to access Elastic with .NET: during my first few experiments I felt like I was “wrestling” the API.

2. Gulp, or: More Gulp!

I’ve done quite some work with Gulp recently, and it was a very (and surprisingly) fun experience! In the past, most devops tasks felt like chores to me, but not so much with this Gulp. So this point would actually be “More Gulp” (as I’ve done quite a bit of it recently already), but I still have quite a list of things I want to try out, like for instance setting up JSCS.

3. Node

Yes, yes, I know: I’d be a hipster-after-the-fact if I’d start on Node now. Even so, I enjoy writing Javascript, and Node would be a way to do even more of it in my pet projects. Not sure how I would be utilizing Node yet: with ExpressJS, for custom (Gulp) packages, general scripting, or perhaps something like project Euler.

4. Linux

This one’s been on my mind for a while now: perhaps I’ve been stuck too much in my Windows-comfort-zone. It would be good to shape up my base knowledge of Linux as a development environment.

5. Grunt

Why yes, in addition to Gulp there’s also Grunt on this list. Gotta know what’s on the other side of the fence to be able to compare, right?


I’m placing a short but distinct separation here, because these are currently mostly “in reserve” kind of ideas.


6. Couchbase

Working a little with this at work as well. There’s a few things I’d be excited about to try at home, mostly features from version 3 and 4. Would need a “subject” though to make it interesting…

7. Study three-way-diff-tools

For some reason I never got my mind wrapped around these tools. And that’s frustrating. I should just spend a few hours trying to understand the details of these tools… I guess.

8. Raspberry / something IoTish

This one’s on the list because it nags me that I don’t see what all the rage with IoT is (or is it “was”?) about, and for that reason I feel like I have to “make Slackbot call a webhook that calls my webserver that calls my Raspberry which turns on a LED strip and autopilots a drone around in my room”. Or something.

9. Android app

I’m curious how the Android-dev-experience compares to my short experiments from around 2011. This would also allow me to revisit Java. But on a whole I feel I’d have to invest a lot of time or not do this one at all.

10. Closure Compiler

It’s on the list because I find the idea of it intrinsically interesting. It’s low on the list because I’m not sure if I work on things personally or professionally at the moment where this would be worth the investment.


Wow, that worked like a charm! Apologies for the stream-of-consciousness post, but now I do know what to do next! And I’m not even going back to adjust the (order of the) list accordingly, I’m just gonna leave you all hanging out there, guessing at what’s next…

All roads lead to Excel, even those from SQL

My first employer provided me with some valuable insight:

Microsoft Excel is the main competition for any piece of software.

Over the years this statement has proven true an alarming number of times. And it makes sense too. Everyone knows how to use Excel, and it’s extremely flexible, especially when you’re working with any kind of (tabular) data. In other words: all roads lead to Excel.

Roads from SQL also often lead to Excel, even though they’re not always pretty. Sure, if you’re on foot or horseback, with a limited amount of luggage, the road will be fine. However, here’s a particular scenario that obscures the path.

Scenario

These are the basic constraints:

  • Available tools: MSSQL 2012, SSRS, SSIS, Visual Studio 2012 & .NET 4.5.
  • Excel versions: either XLS (2003 and below) or XLSX (2007 and up, slightly preferred) will do.
  • Form of data: combination of normalized and denormalized data (see below).
  • Amount of data: tops 250.000 rows (times 20 when unpivoted).
  • Required response time: live exports that should run within seconds.
  • Databases: many instances each with the exact same schema but different data.

So there’s access to the latest and greatest Microsoft tools, and the option to include custom components. Free and open source components are preferred, but buying tools and components is also an option.

Data

Here’s a simplified version of how the data is modeled:

  • Person is a “flat” table, containing some columns that have “fixed” personal details.
  • Property and Value allow for custom (normalized) Person fields.

Here’s a visual of this simplified model:

Database model for Person, Property, and Value

 

You can view the SqlFiddle with some sample data. A typical query to get the data that’s going to be our starting point:

This will give output similar to this:

Id FirstName Surname Id PropName Id ValName CustomValue
1 John Doe 1 Trait 2 Bold NULL
1 John Doe 1 Trait 3 Easygoing NULL
1 John Doe 2 Eye color 4 Green NULL
1 John Doe 3 Pet name 7 Placeholder Fluffybunny
2 Mark Roe 1 Trait 3 Easygoing NULL
3 Mary Fisch 2 Eye color 6 Other… Red-brown-ish
3 Mary Fisch 3 Pet name 7 Placeholder Ravi

Note that in reality I’m dealing with these figures:

  • About 30 columns on the Person table;
  • About 20 different Properties with about 6 possible Values on average;
  • Anywhere between 100 and 250.000 Persons;
  • Usually between 0 and 2 Values per Person per Property;

For one, this means that the normal output of mentioned query has a lot of redundant information (i.e. the 30-ish Person columns).

Target Output

The business requirement here when moving this data to Excel should be obvious: the data should be pivoted. Each “Property” should become a group of columns, with one column per “Value”. A table says more than a thousand words; this is the requested output:

Trait Eye Color Pet name
Id FirstName Surname Bold Easygoing Green Other… Placeholder
1 John Doe x x x Fluffybunny
2 Mark Roe x
3 Mary Fisch Red-brown-ish Ravi

Something along these lines is what the business user’s would like to see.

Bonus Objectives

Getting the target output in itself is a challenge. I’m not done yet though, here are some bonus objectives we have (with MoSCoW indications):

  • Properties and Values both have ordering, the order of columns Should respect that.
  • Any solution Should allow for some styling (fonts, borders, backgrounds. It’d be Nice to have further control, for example enable a theme, alternate row coloring, etc.
  • I Would like to have a place for metadata (date exported, etc) somewhere in the generated file.
  • Localization of the column headers (where applicable) would be Nice to have.
  • It’d be Nice to be able to reuse much of the solution in generating XML files instead of Excel sheets.
  • Any solution Must be solid and maintainable.
  • Any solution Must run on moderate hardware without hogging resources.

Current Solution

Right now, the above is accomplished using Reporting Services. This works decently well for datasets containing no more than a few thousand Person rows with related Property Values.

However, beyond about 3000 records performance quickly starts to degrade. This isn’t entirely unexpected, because Reporting Services isn’t really meant for this task (it’s much better at showing aggregates than at exporting large volumes of data).

Possible Solutions

There are many possible solutions. I’m currently considering a subset of them (some solutions merely for “benchmark” purposes):

  • SSIS packages. The tool seems meant for the job. I do hold a grudge against the tool, but maybe it’s time to get over that.
  • Dynamic SQL + generated RDLs. Use DynSQL to do the pivoting. This requires generated RDL files because the fields of a query must be known up fron to SSRS.
  • Dynamic SQL + OPENROWSET + OleDB. Use DynSQL to do the pivoting, and export it straight to Excel using OleDB.
  • FOR XML queries into OpenXML. The basic idea: fast FOR XML queries, possibly an XSLT, generating OpenXML data, and plug it in a basic XLSX.
  • ORM or ADO.NET into an OpenXML using an XmlWriter. Something along these lines.
  • BCP directly into Excel files. Haven’t looked into this yet, but it may be an option.
  • SQL CLR. Not sure how this would work (if at all), but there might be options here.

Now it’s time for me to try some of them, see which one fits the requirements best.

Programming music

Lying is useless: I’m addicted… to music. Just about whatever I do, I will have music in the background. Especially when I’m working on the computer: programming, photo-shopping, video-editing, gaming all require music.

Different activities and varying times of day require different types of music for me. Luckily, I enjoy just about all kinds of music. The growing popularity of Spotify has driven both di.fm and last.fm radio from my favorites, and allows me to pick just about anything I want to hear.

This brings us back to the topic of this post then (hope no-one was expecting a guide on how to program music on some device), music for programming duties! I realized I put on certain types of music whenever I’m in a certain type of zone. So here’s a go at my music choice per programming task.

Task Music Genre
Creating C# interfaces, designing server side code Dubstep or Classical Music
C# code (or any server side code, for that matter) Any album that’s very familiar, e.g. Homework
Database design & SQL queries (SELECTs) Electro, Hardstyle, or even some old Gabber tunes
SQL UPDATEs and DELETEs Ambient or downbeat
CSS and HTML (skinning and the like) 90s dance or dirty house
Debugging a difficult problem Classical music or Gregorian chants
Creating wireframes More dubstep
Excel formulas and VBA macros Death Metal or Industrial Powernoise
Footnote: recently ghettofunk has been a great fallback for any programming task. Stickybuds!

Hopefully I’ll be able to look back at this post some time in the future and create an updated overview. Perhaps this will even inspire someone to up a music-programming cross table of their own. If you do: let me know!