Git intro for TFVC users

So, you’re using Team Foundation Version Control (TFVC). You know about Team Projects and Collections, have a “stable” and “dev” branch for most projects, know how to do basic merges, and know how to shelve and unshelve changes. But you only have superficial knowledge of Git.

Well, then you’ve come to the right place: my Git intro for TFVC users.

Disclaimers

Here’s a heads-up about this particular post:

  • It is not in-depth;
  • It will oversimplify things to the point where not every statement is technically true;
  • It does not teach you actual Git skills or commands;
  • It is somewhat subjective.

Oh, and in my humble opinion: Git has a crazy learning curve. So buckle up!

And, as a final disclaimer: I’m writing this because I want to be able to explain the basics of Git, and certainly not because I’m an expert. In fact, I know more about Mercurial than about Git, I have worked mostly with TFVC in the past year, and would have to look up nearly every Git command when using the command line (I prefer GUI tools most of the time). Just so you know.

Disclaimers done, let’s get started!

On “TFS” vs “TFVC”

First, let’s get the TF* terminology right. These terms are different but related things:

  • “TFVC” stands for Team Foundation Version Control, and is the actual system for keeping history of your codebase;
  • “TFS” stands for Team Foundation Server and is the “environment” (if you will) in which source control features come together.

Many people, myself included, often use the acronym “TFS” when we actually mean “TFS with TFVC”. This is probably because TFS is very often used with TFVC as the version control system; but note that you can also use Git with TFS.

This blog post focuses on TFVC (which almost always implies you’re using TFS too).

Why Learn about git?

The personal reasons for learning Git are quite simple: it’s the de facto standard for version control. Knowing about Git is:

  • crucial for your career (your new employer is likely to use Git);
  • crucial for talking to new hires (who will very likely know Git);
  • crucial for efficiently navigating open source.

The intrinsic reasons for learning Git all come down to the fact that it is insanely powerful (which also accounts for the steep learning curve). After having worked with TFVC in a 12 person team for over a year, I’d like to highlight the following main- and most direct advantages over TFVC:

  • Cheap branching;
  • Small repository size;
  • Local commits;
  • Better options for “shelving”;
  • Speed;
  • Better “offline” support.

Beyond these advantages, which you’ll get if you’re using a “central” Git repository, there are even more goodies if you tap into the distributed nature of Git, as well as the more powerful commands (e.g. to rewrite history).

Basic differences

Git is a lot like TFVC, in that it is also a Version Control System (VCS). It is also quite different. Here’s how they compare when managing the code base and doing changes:

TFVC Git
There is a central, server-hosted Team Project and you’ll have one or more local workspaces containing a copy of all the code. For now, let’s assume that there is a central repository hosted somewhere on a server. You’ll have one or more local “clone” repositories containing a copy of all the code and also all of its history.
check in is a command to send your pending local changes to the server where they will be committed to the version control history. You commit changes locally which creates a “change set” private to your “clone”. You can do this multiple times. You push one or more change sets at once to the central repository.
You can Get the Latest Version from the server which directly tries to merge with your current state. You fetch changes from the central repository and merge with your current state (or do both at once by doing a pull) possibly creating a new commit.

The fact that a TFVC “check in” is multiple separate commands in Git gives several advantages:

  • You can build history in small, individual steps (commits), which you can undo individually in several ways, and you can apply individual commits to other branches.
  • Your changes are “safe” in commits when you want to send your changes to the central repository and find out others have conflicting changes, by default they won’t get “lost” if merges go bad. With TFVC, you’re required to resolve conflicts as you try to check in, which can screw up “unsaved” changes.

Our next set of differences would be about branching, but before that it’s good to talk briefly about “shelving” changes.

Setting changes aside

With TFVC you can shelve changes you currently have to safeguard them, optionally reverting those changes in your local workspace. You don’t necessarily need to do this when switching work to another branch, because the branch is typically a sibling folder of the main branch. More on that later.

With Git, you can stash changes you currently have, reverting changes in your local clone. You have to do this before switching to another branch. In order to see how that works, let me move on to the next topic: branches.

Branches

A typical setup in TFVC starts like this (with a local workspace matching the Team Project 1-on-1 in this example):

Inside the project folder for the Team Project, there immediately is a sub folder called “main”. This is done so that it is easy to branch off the entire codebase for that project into this:

The folder “dev” is now a complete copy of “main”. This also means you could be working on two branches simultaneously, and you can check in files to both branches at ones if you so desire. You could even have “main” and “dev” at different points in your version control history.

Git is different.

With Git, the “main” folder as you’d have it in TFVC would be pointless. Instead, there is just:

You can “branch off” any state of “project-x” at any point in time. Your folder “project-x” will point at a certain state, de facto having the code for a specific branch.

If you would like to have both branches “ready for action” on your disk, you would typically have multiple “clones” of the repository. One would be at the most recent state of the “main” (typically called “master”) branch, and another might be at the most recent state of another branch.

As a summary, Git branches relative to TFVC branches:

  • Are light-weight and can be used to “switch context” for example to work on a feature;
  • Are a top-level thing, as opposed to the “path-based” branches in TFVC;
  • Allow for more precise merges and “cherry picking” of commits;

These differences allow for completely different workflows with Git. This is a topic for another post though, if you’re interested I’d recommend research things like “Git Flow”, “GitHub Flow”, and “GitLab Flow”.

Other differences

While there are many other differences, both small (e.g. specific commands) and big (e.g. the “distributed” nature of Git), this post will keep it at the above. Mentioned differences are in my opinion the most “direct” and eye-catching differences. If you want to learn about the other differences I recommend you first get started with some practical skills, e.g. by following a tutorial.

Conclusion

TFVC is a mature, decent version control system. But personally, having worked with both centralized and distributed version control, in TFVC I’m misssing:

  • Ability to do commits locally;
  • Cheap, more powerful branching;
  • Great “shelve” options;

Those three, in addition to its “distributed” nature enabling online platforms like GitHub to flourish, are likely the reasons Git is so popular nowadays. In my opinion those are great intrinsic reasons to learn Git when you’re currently using TFVC (even if you cannot switch on the short term), and if not for those reasons then because it’s crucial for your career.

So ready yourself for a bad-ass learning-curve climb, and start learning more about Git!


Resources & Further Reading

Here are some links to continue your journey:

Pet project status report

This post assumes you’ve read my previous post on this project. It’s going to be a very short status report.

In aforementioned post I’ve tried to break through my analysis paralysis by listing all the things I had to think about (it helped!). Let me re-iterate, update, and complete the list to once again gather my thoughts:

  1.  Project and Namespace structure. KISS, so I went with just a single project (plus one for tests) for now.
  2.  Folder structure. following the lead of popular C# projects, only needing some files in the root and a src folder for the projects.
  3.  Initializing git. this was actually the biggest mental blockade. I just bursted through: screw TFS history, screw being “optimal”, and just move. I copy-pasted all code, cleaned everything carefully, and initialized the git repo with an already decently sized project.
  4.  License. MIT.
  5.  GitHub setup. Let’s start simple. Project created under our organization’s GitHub account. Pushing straight there for now, using my own personal profile. Will think about working with forks and pull requests later.
  6. NuGet packaging. Haven’t started this yet.
  7. Re-including the open sourced bits in my closed source solution. Have postponed optimizing this. The not-so-optimal solution for now is that the projects are gone from TFS, and there’s a “lib” folder instead with compiled DLLs from the open source project.
  8.  Choosing a name. Chosen, but not ready to disclose yet, even though it’s very easy to sherlock this bit.
  9. .NET Core. We’ll cross that bridge when we get there.
  10.  What am I forgetting? Looking at some of the “top” C# GitHub projects (Restsharp, NodaTime, dapper-dot-net, AutoMapper) was extremely helpful. Note to self: in-depth code reviews of those projects will be extremely educational.
  11.  Minimum quality. I actually chose to make this project an exercise in being as “clean” as possible (within -though close to the edge of- reason). But one bullet at a time, so e.g. crafting a great readme is a sub-exercise left for later.
  12. Early feedback. Still on my list, but want to get to some kind of “alpha” stage before I send out review requests.
  13. Logo. Was a great excuse to get started with the recently released GitHub Projects feature.
  14.  XML Documentation. Probably way over the top, but a nice personal exercise, so worth it after all.
  15. CI. Have not started with this yet, but know I have to at some point.
  16. GitHub Wiki. Probably way over the top, but would be a nice personal exercise to create one.
  17. Domain name. Not sure how that plays together with the license, the fact that the repo was initialized under my organization’s account, or any trademark stuff. Will have to figure that out some time soon.
  18. .NET Framework Versions. Related to the .NET Core bullet I guess, but slightly more important. I found that the popular repo’s I looked at for guidance have some kind of setup with duplicated projects several times over, not sure how that works. For now I’ll have to stick with a (unfortunately slightly older) version, 4.5.1, because that is the highest version I can use in the project that is dog-food-testing the project.

Okay, now I can stop that hurricane of thoughts, and get back to this pet project, and tick some more things off the list!

Extracting and Publishing closed bits as Open Source

There’s a project at work that’s got all the traits to serve as a great vehicle for dabbling some more with open source:

  • Small-but-not-too-small;
  • Self-contained;
  • Not too specific (e.g. to our domain);
  • Keeping the IP safe isn’t a concern since it’s just a small tool unrelated to the core business (i.e. we’ve got “the okay” to make it open source);
  • It might be of some use to someone else, and failing that it’s still fun to open source it;

While trying to start with the process of extraction and open sourcing, there seem to come up a million things to think about when going through this process. I want to gather my thoughts (by writing this post) and untangle that mental mess, so that I can get to a clear plan of attack. Perhaps this post will even be of use to someone else, somewhere.

Context

Currently, the bits I want to make open source are only slightly coupled to the bits I want to keep closed. That is: currently everything is in one solution, but the split in projects already clearly demarcates what would be open and what would remain closed. To put this in a tabular overview (where “Foo” is the closed bit, and “Bar” is the open bit):

Current Situation

Closed Source

/Foo.App
/Foo.App.Tests
/Foo.Core
/Foo.Core.Tests
/Foo.MyInstances
/Foo.ConsoleWrapper
/Foo.ServiceWrapper

Target Situation

Open Source

/src/Bar.App ???
/src/Bar.App.Tests ???
/src/Bar.Core
/src/Bar.Core.Tests
/src/Bar.ConsoleWrapper
/src/Bar.ServiceWrapper

Closed Source

/packages/Bar ???
/Foo.MyInstances

/Foo.WindowsService ???
/Foo.Console

There are some question marks in that overview. Elaborating on them a bit:

  • I’m not sure if the distinction between “Core” and “App” should remain a dll/project splitup as opposed to possibly a namespace split.
  • I’m not sure how the Open Source solution will be included in the closed source app (via NuGet, or as external source or binaries, or…?).
  • I’m not sure if I can easily distribute a generic windows service wrapper, or if I need to create it in the closed source solution after all.

In addition to these source-related questions, there are other things to think about as well. So next up:

What things to think about beforehand?

Let’s summarize all the things I can currently come up with that might be important in this process, in no particular order, merely numbered to be able to reference them easily:

  1. Project, Namespace, and DLL structure: what’s wise, what’s neat, what’s useful?
  2. Folder structure: how will the repository be structured? What’s future proof here?
  3. Initializing git: currently, the project history is in our corporate TFS. So not even sure if/how to keep the history intact, or if that’s even feasible.
  4. License: okay, that’s not too difficult, but have to choose one nonetheless.
  5. GitHub setup. What’s a good setup? Should I make the organization main owner of the repo, with a personal fork from which I do pull requests?
  6. NuGet packaging: how does this even work? As an application developer I had never had the need to learn how any of this works.
  7. Re-including the open sourced bits in my closed source solution: via NuGet, as external source, or as external binaries?
  8. Choosing a name. One which is not already in use for some software, where a domain with “normal” extension is still avaible, where the name is not taken on GitHub yet, etc.
  9. .NET Core. From my rss feeds I gather that it can be a quite fuss to make open source projects .NET Core “proof”, but it would be yet another thing to tackle while starting this whole thing off. Perhaps something for later? Or is that a grave mistake?
  10. What am I forgetting? I’ve been looking at some of the “top” C# GitHub projects (Restsharp, NodaTime, dapper-dot-net, AutoMapper) and their git history to see how they have evolved, to find out if I’m forgetting anything.
  11. Decent starting point, code-wise: I might be setting unreasonably (and unnecessarily) high standards for the initial commit, but still something to consider. What is the minimum quality I’m requiring before opening the source?
  12. Early feedback: it might be useful to get friends and colleagues to review the setup and get the first version(s) of the repository just right.
  13. Logo: yeah, this project needs the best. logo. ever!

In addition, there’s a practical point: once I split off bits to an open source project, I’ll effectively have two (source-control-wise unrelated) forks of the same code-base, until I’m ready to make my closed source solution depend on the open source bits again.

In Conclusion

After writing the above: wew! No wonder I was not getting anywhere: my mind was just wandering in circles through all of the above points. And I guess “Perfect is the enemy of Good”, or some such. Time to tick off some of the above items.

Aggregated online interactions

This blog hasn’t seen much action lately, but that’s a misrepresentation of my online interactions. Most of my interaction in the past few months has been on Stack Overflow Q&A, and some on Stack Overflow “Documentation” as well as a small amount on GitHub. I wanted to aggregate some of those interactions on my blog, as well as perhaps cross-post bits and pieces here, mainly for my own reference.

Let’s start with the first: aggregating the bits and pieces that I want to have easy links to.

Stack Overflow Documentation

  • Showcasing all common Angular constructs“. I’m linking to the most up to date version. I wrote V1 of that article, which was subsequently improved by various other folks. It’s the tutorial (and equally important: the style of tutorial) I wish I’d had when I started learning Angular.
  • KnockoutJS “Equivalents of AngularJS bindings“. Linked page summarizes the state SO Documentation is currently in, at least for low-traffic tags: little and poor collaboration, and some frustration because some decent examples I wrote just don’t get reviewed (neither approved nor rejected). Thinking I might turn my content there into a (series of) blog post(s) here. Not sure yet.

Stack Exchange Q&A

At around 20 questions and 200 answers in 2016 so far I’d say I’m “moderately active”. Here’s a few that stood out when I browsed through my recent history:

I also gave SoftwareRecs.SE another shot, posting some questions, but they fit right into my question history: lots of unanswered tumbleweeds. And not for lack of trying, as I spend a lot of effort on making my questions there as good as they can be. The main reason I do that (and the reason I keep coming back to softwarerecs.se, in spite of the tumbleweed-factor) is that thinking carefully about your wishes and requirements at the least will help you find something yourself, if no-one else recommends anything.

And even though I haven’t interacted with Cooking.SE much lately, every stray upvote now and then to my “Cooking fish in a dishwasher” answer makes me smile.

GitHub

I don’t interact as much here yet as I’d like. I specifically wish I remembered more often than a measly four times to create gists, because the ones I did create are ones I tend to go back to. In addition to gists, I’ve gotten to creating only very few issues and pull requests, something I want to work on.

One shoutout by the way to the DefinitelyTyped repository, because that community has to deal with a really scattered committer base, and seem to do so pretty well. My pull request (though small) was reviewed and merged quite quickly.

In Closing…

What to do next? The tags I followed on Stack Overflow for answering seem to have dried up a bit. Perhaps some more interaction on GitHub, as well as re-editing some of the above links into blog posts? Then again, a few weeks of vacation to Hawaii are coming up as well, so it might be a while again before posting…

Nested Elastic Explorations – Part 2

Reusing a properly modelled domain for storing data in Elasticsearch does not work well out of the box. Let’s examine a problem scenario. Consider this mini-domain:

Mini Bieb Domain

This ties in with my last post, where I mentioned that loops are a pain for serializing to json. Here’s the loop, visualized:

Mini Bieb Domain Loop

The problem is that NewtonSoft (used under the hood by Nest) will start serializing “The Greatest Book”, and recurses through all properties. In the end it’ll try to serialize “The Greatest Book” again as part of “Richard Roe”‘s AuthoredBooks property.

Breaking this serialization loop is actually pretty simple with NewtonSoft, and since a while you can inject the appropriate NewtonSoft setting in Nest as well. Something like this:

Problem solved, right? Not so much. Here’s why. Suppose I use the LoopHandling “fix” and load up the mini-domain with this integration test:

This will create a document in Elasticsearch of a whopping 71 KB / 1364 lines, see this example JSON file. Not so good.

The simple solution which would do for now would be to index only Book items, and all related people (authors, editors, translators), but not those people’s Books (AuthoredBooks, etc). We somehow need to let Nest and Elasticsearch know that we want to stop recursion right there. The question is how to be explicit about how they should map my domain objects to documents. I see two courses of action I like:

  1. Declarative mapping, with Attributes. This would (to my taste) require separate DTO classes to represent the documents in Elasticsearch, and have an explicit transformation between those DTO’s and my Domain objects. (I wouldn’t like to litter my domain object classes with persistence-specific attributes.)
  2. Mapping by code. This would seemingly allow me to keep using domain object classes for persistance, having the “Mappings” in code as a strategy for the transformation in separate files. At this point though I’m unsure if this approach will “hold up” once you start adding more complex properties and logic to domain objects.

I lean towards option 1, even though it feels like it’ll be more work. Guess there’s only one way to find out…

Nested Elastic Explorations – Part 1

After my previous post on what to explore next I’ve dabbled with option 4 “Linux” (but got frustrated so I set it aside), worked some on option 2 “More Gulp!” (bu that was mostly at work). However, I think I’ve settled for now with putting some effort in learning Elasticsearch and Nest. So let’s start a series of posts on this venture. (No promises though, this “Part 1” post might end up being the only part…)

About this venture…

Elasticsearch can be seen as a NoSQL database, more specifically as a document database. I’ll be using my Bieb pet project (or at least its domain) for testing its features. Bieb’s domain should be familiar to everyone: books!

Specifically I’m writing this post because I quickly ran into a fundamental challenge with document databases, and I hope to gather my thoughts on the matter by writing about it. To set the stage, I’ll be talking about this part of the domain:

  • A Book has multiple Authors (of type Person).
  • A Person has authored mulitple Books.

The classic many-to-many relationship example. The challenge then obviously is how to express this in the various tools.

Setting the coding stage…

Before I dive into the Elasticsearch bit, let me first describe how I got this going in C#, SQL, and NHibernate. First, the SQL bit:

Run-of-the-mill stuff, with a many-to-many table. Obviously, we don’t want to see BookAuthor show up as a class in our code. That is, we want this kind of C#:

As you can see, the Book is the “main” entity and the “boss” of the relationship with authors. Apart from the virtual keyword everywhere, little to no SQL or NHibernate know-how leaked into this domain.

The NHibernate mapping looks like this:

The important bit, and the link to Elasticsearch, is the inverse="true" bit. With NHibernate, you have to indicate which entity is “in charge” of updating the many-to-many table. With document databases you have to do the same thing.

Moving to Nest & Elasticsearch…

Diving head first into throwing these entities into Elasticsearch, I tried to run the following Nest integration test:

It’ll fail, because Nest internally uses JSON.Net to serialize the Book, which means you’ll get this:

Newtonsoft.Json.JsonSerializationException : Self referencing loop detected with type ‘Bieb.Domain.Entities.Book’. Path ‘allAuthors[0].authoredBooks’.

A problem that was to be expected.

With SQL self-referencing loops are just fine, handled by the many-to-many table. In NHibernate it’s a minor nuisance, simply handled by the inverse="true" bit. However, with document databases this is somewhat less trivial, in my opinion. In fact, it may be the hardest part of using them: how do you design your documents?

What I’d like to do…

The solution obviously is to break the self-referencing loop somewhere. Simply not serializing a Person’s authored books should do the trick. (On a side note, it occurs to me that graph databases would not need this trick, and would be great for a scenario with endless Book-Person-Book-… traversal, but that’s for another time.)

However, I don’t want to hard-code the loop-break into my domain object, e.g. by marking AuthoredBooks as not-serializable. The first and foremost reason is that this would mean my storage layer leaks into my domain layer. The second reason is that I may want to break the loop at different places depending on the Elasticsearch index I’m targeting. That is, I’ll probably have an index for Authors as well as Books, and want to break the loop in different places on either occasion.

Questions…

So some questions remain. How could we do this with JSON.Net without altering the domain layer? Or is there a way to do this with Elasticsearch + Nest? Or should we bite the bullet and have a seperate set of DTO’s to/from our domain entities to represent how they’re persisted in Elasticsearch? Or do we need a different approach alltogether?

Good questions. Time to go and find out!

A New Hope

Did I mention before that I write here to collect my thoughts and as writing practice? Well, in any case, after my previous monster post, and given that it’s the start of a new year, I figured I’d do a “New Beginnings” post. Then the sci-fi addict in me awoke and typed “A New Hope“, even though I’m a Trekkie at heart. But I digress.

I wanted to summarize which technologies and experiments I’m most interested in pursuing next. Let me try to list them in order of current preference.

1. Elasticsearch …. and Nest?

Elastic as a (primary?) data store intrigues me. It feels like a second chance for Lucene towards me (experimenting with Solr didn’t “click” for me). There recently was a major version released so it seems like a great time to start investigating. I’m hesitant about Nest though, which you’d kind-of have to use if you want to access Elastic with .NET: during my first few experiments I felt like I was “wrestling” the API.

2. Gulp, or: More Gulp!

I’ve done quite some work with Gulp recently, and it was a very (and surprisingly) fun experience! In the past, most devops tasks felt like chores to me, but not so much with this Gulp. So this point would actually be “More Gulp” (as I’ve done quite a bit of it recently already), but I still have quite a list of things I want to try out, like for instance setting up JSCS.

3. Node

Yes, yes, I know: I’d be a hipster-after-the-fact if I’d start on Node now. Even so, I enjoy writing Javascript, and Node would be a way to do even more of it in my pet projects. Not sure how I would be utilizing Node yet: with ExpressJS, for custom (Gulp) packages, general scripting, or perhaps something like project Euler.

4. Linux

This one’s been on my mind for a while now: perhaps I’ve been stuck too much in my Windows-comfort-zone. It would be good to shape up my base knowledge of Linux as a development environment.

5. Grunt

Why yes, in addition to Gulp there’s also Grunt on this list. Gotta know what’s on the other side of the fence to be able to compare, right?


I’m placing a short but distinct separation here, because these are currently mostly “in reserve” kind of ideas.


6. Couchbase

Working a little with this at work as well. There’s a few things I’d be excited about to try at home, mostly features from version 3 and 4. Would need a “subject” though to make it interesting…

7. Study three-way-diff-tools

For some reason I never got my mind wrapped around these tools. And that’s frustrating. I should just spend a few hours trying to understand the details of these tools… I guess.

8. Raspberry / something IoTish

This one’s on the list because it nags me that I don’t see what all the rage with IoT is (or is it “was”?) about, and for that reason I feel like I have to “make Slackbot call a webhook that calls my webserver that calls my Raspberry which turns on a LED strip and autopilots a drone around in my room”. Or something.

9. Android app

I’m curious how the Android-dev-experience compares to my short experiments from around 2011. This would also allow me to revisit Java. But on a whole I feel I’d have to invest a lot of time or not do this one at all.

10. Closure Compiler

It’s on the list because I find the idea of it intrinsically interesting. It’s low on the list because I’m not sure if I work on things personally or professionally at the moment where this would be worth the investment.


Wow, that worked like a charm! Apologies for the stream-of-consciousness post, but now I do know what to do next! And I’m not even going back to adjust the (order of the) list accordingly, I’m just gonna leave you all hanging out there, guessing at what’s next…

Finishing Bieb

I’m calling it. Bieb is now officially wrapped up.

I’ve accomplished the bare minimum I set out to do with Bieb, and learned a lot of extra things along the way. But motivation to continue any further has dropped to near zero. And I’m okay with that.

Fair warning: this is a long post. Consider the above to be the TLDR version, and whatever’s below as a peek into my personal notes and thoughts. Read on at your own risk!

Still here? Allright, here goes.

Conclusion

So let’s start up with a preliminary conclusion annex Table of Contents:

  • Bieb was a fun project. Now it’s “wrapped up” / abandoned.
  • Azure sucks I disliked working with Azure for a hobby project. Great tech, but that learning curve…
  • Proper errors in ASP.NET MVC are hard.
  • ASP.NET MVC is a nice framework in general though.
  • Database integration tests for NHibernate are useful and easy to set up.
  • Bieb is a successfull TDD experiment.
  • Razor is great for static content. You need nearly zero JavaScript for views.
  • Designing (the domain logic for) an entire application is fun!

You want more details you say!? You shall have them! Moving on…

Elevator Pitch

First up, the elevator pitch. There actually was something of the kind (though not in typical pitch-format) on the project home page:

Website project based on ASP.NET MVC for managing and displaying your personal book collection on the web.

At the bottom of the page, the (equally important) secondary objectives are mentioned:

  • Finding out more about Codeplex
  • Experimentation sandbox for MSSQL, ASP.NET MVC, NHibernate, Ninject, html5, css3, jQuery, and Modernizr
  • Experimentation sandbox for trying out competing and additional frameworks
  • Having this website project available for anyone who’s interested
  • Real-life events such as BBQ’s and drinks to “discuss the project”

And that was actually pretty close to what it turned out to be.

Note that Bieb is not solving a new problem, nor is it solving a problem in new ways. There’s GoodReads, which is a great site that does 90% what Bieb does and more. Bieb was not meant for us to get rich with a “next big thing”; it was meant as a fun project to toy with technology.

The Negatives

Before I get to the positive things about this project, first some negatives.

Azure is no friend of mine

Bieb had to be a replacement for a version of this app I’d hacked together in PHP. With PHP however, it’s very easy and cheap to get some hosting, and very easy to set up a deployment. With .NET, there is no such luxury. Private / shared hosting of .NET sites is expensive. The best option seemed to be Azure (as I had a MSDN subscription that comes along with some Azure credits), but Azure has a very steep learning curve compared to getting a PHP site up and running.

Put in another way: Azure feels solid, but it also feels big and “enterprisey”. There’s just so many buttons and settings and “thingies” that it feels impossible to get started. Watching tutorials doesn’t help much, because your choices are (a) a very specific 20 min tutorial that doesn’t cover all your needs or (b) watching a 6-hour Pluralsight course and still being covered only 80% of the way. Add to that the fact that it’s still under heavy development (which mostly is a good thing, but doesn’t make the learning curve any better), and you’re set up for frustration.

One particular anecdote worth mentioning is as follows:

  1. I tweeted that I found it confusing that one Azure portal redirects me to yet another Azure portal.
  2. Azure support tweets back redirecting me to MSDN forums.
  3. In the forum thread, I get redirected to yet another forum where I should post my findings.

I think I would now like to redirect Azure to a place where the sun don’t shine.

Don’t get me wrong though: I love .NET development and Windows hosting in general. Both Azure and hosting yourself are fine options for business-type projects. However, I’m deeply disappointed in using Azure and .NET for web hobby projects. It’s probably the main reason I’m abandoning wrapping up Bieb, and the main reason I’ll be focussing on other tech for hobby projects in the near future.

Great error pages with MVC are hard

A very specific “negative”, but one that annoyed me to no end. I think I’ve tried to attack this problem 5 times over, and failed every single time.

This is what I wanted to accomplish:

  • No YSODs. Ever. Making sure that if your app fails it does so elegantly is very important IMHO.
  • Proper HTTP status codes. That means 404s for non-existent static resources, MVC routes that can’t be resolved, but also “semantic” 404s for when a domain item (e.g. a “Book” or an “Author”) was not found.
  • A nice error page. That means a proper MVC page when possible, one that gives options (like a Search partial view) and relevant info. Failing that, a nice, styled static html page.
  • Proper error logging. The logging and error handling code should be unobtrusive to the business logic. If a request fails (i.e. a 500 error) it should be caught, logged, and the user should be directed to a meaningful, useful page with proper content but (again) also a proper HTTP status code.

Whenever I tried to tackle these requirements, I would fail to do so, and clicking through the online resources about this problem I would inevitably end up at Ben Foster’s blog post on this problem. That excellent post notwithstanding, I have never gotten this to work on IIS or IIS Express.

And having failed to get this working locally, I dread the thought of having to get this to work on Azure…

The Positives

There were also many positive aspects about fiddling with Bieb. Here are the main ones.

NHibernate Database Integration Tests

I continued to work on a way to run database integration tests with a unit testing framework, based off an approach we took at my previous job. Here’s what the base test fixture looks like:

The Factory has some more bootstrapping code for recreating the database schema from scratch. All this setup allows you to write this kind of test:

Which is a nice way to fix a database issue in a test-driven manner.

Currently my approach has one big flaw though: the tests are not isolated. Each test has to account for previous tests possibly having left data in the database. For the tests written so far that doesn’t matter, but it’s an accident waiting to happen nonetheless.

I see basically two solutions for this:

  1. Drop and recreate the database before each test. But that’s probably slow.
  2. Wrap each test in a transaction, and roll it back at the end. But that exludes the option of testing things that require actually committing transactions.

Truth be told, I wrote most database integration tests because I was unsure of how NHibernate would function and/or handle my mappings. And for that (i.e. learning NHibernate) the current set up worked just fine. If I were to continue with Bieb there would most likely come a time where I’d go for option 2. I think Jimmy Bogard typically advocates a similar approach.

PS. For what it’s worth, I’ve written a GitHub Gist with a minimal setup for creating NUnit+NHibernate tests, specifically geared to creating minimal repro’s to share with colleagues or on Stack Overflow.

ASP.NET MVC is a very nice framework

I quite like the kind of code you have to write to get pages to the user with MVC. Here’s an example of a Controller action in Bieb:

It’s short and to the point as it delegates the other responsibilities to (injected) dependencies like the Mapper and Repository. In addition, Razor views are also pretty easy to write. At the least, they are a breath of fresh air after WebForms.

So, if you want to create web sites with a big C# component, ASP.NET MVC is a fine choice. However, I do ponder if a more SPA-like approach (with Web API or similar + an MV* client side library) or a full-stack JavaScript solution are better choices for new web development projects…

Having said that, I’m very pleased with how Bieb was a great way to learn ASP.NET MVC.

Test-Driven Development

Bieb Code Distribution

I went to great lengths to stick to TDD development, even when things got ugly. In particular, I’ve had to learn how to deal with several awful dependencies:

  • Random
  • HttpContext
  • HtmlHelper (and all the things it drags along)

To show the length I went, here’s a summary of the Visual Studio code analysis for LoC:

  • 49% (1.564 lines of code) for unit tests
  • 8% (253 lines of code) for database tests
  • 43% (1.371 lines of code) for all the rest

Even though Lines of Code is hardly ever a good metric (it would be easy to inflate these numbers one way or the other), in this case they do reflect the actual state of the code base. Note that this does not include view code (obviously), nor does it include client side code (there isn’t nearly any; more on that later).

Looking back at the code, I’m also pretty pleased with how the tests turned out. There’s very simple, early tests like this one:

As well as tests that border on not being unit tests anymore, but serving IMHO a great purpose:

Which tests your haven’t forgotten any attributes like these:

Which is worth the fact that it’s not a “clean” AAA-style unit test, in my opinion.

Where’s the client side code!?

But wait a second: where is the JavaScript? The answer: there isn’t (nearly) any! All the custom JavaScript for Bieb is this inline bit of code:

This may seem silly anno 2015, and truth be told: it is!

Bieb was a vehicle for me to learn mostly server side tech. Most of my hobby projects, as well as the larger part of my day job includes client side programming. I’m confident I could improve Bieb (a lot) with rich client interaction, but haven’t found the need to do so (yet). I was aiming to get a 1.0 release out with mostly MVC code, and work on improved client side components after that.

One important thing to note: I was very pleased with how far you could get with just Razor in creating html. I was also pleased with how much fun it was to generate static content, which Bieb is mostly about.

The admin views of Bieb suffered most from the lack of JavaScript. The UX is bad, at times even terrible. However, with me wrapping up this project that will likely never change.

How are things wrapped up?

So, how are things wrapped up? Well:

  • You can check out bieb.azurewebsites.net, at least as long as my free Azure credits last.
  • The CodePlex Project will go into hibernation, but will remain available for as long as the wise folks at Microsoft keep it up and running.
  • Some screenshots can be found at the bottom of this post.

And that’s that. Which brings me to say something…

In conclusion

Bieb was a great learning experience. There were things (apart from aforementioned struggle with Azure) I did not really enjoy coding, including:

  • Setting up Logging. A real website needs this, but setting it up is a chore. Setting it up so that it’s unobtrusive can even make it a tricky chore.
  • Setting up a Dependency Injection Framework. DI itself can be a great help, I find that it kept my code clean and testable by default, but choosing- and learning how to use a specific DI container didn’t feel particularly interesting.
  • A proper Unit of Work pattern. It’s necessary, but certainly not my favorite bit of coding. And it probably shows in my codebase, too.

But this is vastly outweighed by the things I did enjoy coding:

  • Razor Views were fun to write.
  • Designing the Domain and its logic was fun, even though unfortunately sometimes the underlying persistance layer leaks through.
  • Project and Solution structure: fun things to think about, even though Bieb is just a small application.
  • Routing: it feels good to have nice, pretty URLs.
  • Controllers: because I did spend some time on the things I did not like, the controllers did end up looking pretty good.
  • Database structure, written through NHibernate mappings that actually is used to generate the database from code.
  • Design. Truth be told, the design is at most 49% mine, but that doesn’t make me any less proud of the end result.

Bieb was a fun project to do. I learned a lot. And even though I had many more plans, and even the start of a big backlog, it’s time to move on.

Farewell Bieb, slumber in peace!


Bieb - Home - LG View

Bieb - Book Index - LG View

Bieb - Book Details - LG View

Bieb - Book Details - XS View

Bieb - Book Edit - LG View

Bieb - Person Index - LG View

Bieb - Person Details - LG View

Bieb - Person Details - XS View

Bieb - Series Index - LG View

Bieb - Series Details - LG View

Bieb - Publisher Index - LG View

Bieb - Publisher Details - LG View

Bieb - Search - LG View

Finishing Things – Intermezzo part 2 of 2

So, like I wrote, Google is “bidding farewell to Google Code“. So my recently “wrapped” up projects needed some post-wrapping-wrap-up (you still with me?!). I’m using this as a good excuse to partially move from Mercurial to Git, because I suspect Google Code’s export to GitHub is probably the migration path they’ve put most effort in.

So, here’s my first (active) day at GitHub:

GitHub Contributions

I’m a little afraid that -given GitHub’s popularity- many will see this and consider it to be my current status on code hosting services (disregarding commits to those repo’s being done over a larger time span, and disregarding activity on CodePlex). However -given GitHub’s popularity- I guess my GitHub activity will increase anyways over the coming years.

As a related side note: moving projects from Google Code to GitHub was extremely easy. The web-based tool required nothing besides a link to my Google Code project, and me authorizing Google Code in GitHub through OAuth. The only downsides I personally found during this automated conversion:

  1. My old commits in the migrated GitHub repo’s don’t get attributed to my GitHub account, because the user details were different. Also, apparently TortoiseHg “helped” me commit unwittingly as “jeroenheijmans <jeroenheijmans@localhost>”. I guess if I’d payed attention I (sh/w)ould’ve at least used a valid e-mail address. (Yes, given that probably no one has cloned my repo’s (yet) I could probably rewrite history to change this, but it’s not that big a deal I guess…)
  2. My repository includes some Mercurial-specific files, i.e. a .hgignore file. I might’ve stripped that if I had done the Hg-Git conversion manually.
  3. The project home page from Google Code was not automatically added as a README.md file in the GitHub repo. I think that would’ve been nice. Then again, this omission gave me a chance to check out Git(Hub).

And so my move to Git(Hub) begins. I’ve already enjoyed reading through other people’s thoughts on (Git) commit messages, had my first dealings with Atlassian SourceTree, as well as a combination of the two. You’d almost wonder how I got the actual migration done at all.

I feel excited about getting to know more about SourceTree, Git, and GitHub. However, I’m also still very determined to finish Bieb, which will remain on CodePlex. We’ll see how that plays out in the coming weeks / months. Stay tuned.

Finishing Things – Intermezzo part 1 of 2

In the beginning of 2015 I declared I was wrapping up my open projects. I’ve since quickly wrapped up BattleTop, TimeLine, and DotaGrid. After that I’ve started to work on my somewhat bigger project: Bieb. I haven’t had as much time to put into that project as I wanted, but there has been a thin stream of commits.

However, all this has been brutally interjected by Google: they’re “bidding farewell to Google Code“.

Google Code was very barebones, but in a sense that was its charm. I like BitBucket and CodePlex too, but for small projects Google Code was just fine. Especially since I’ve shut down those projects. However, now I need to make a choice: do I put effort in moving my discontinued projects? And if so, do I move them to another Hg provider, or do I bite the bullet and convert them to Git?

Guess I’ll bite the bullet. Let me do so right now, and get back in a fresh post with the results…