If you're new here, you may want to subscribe to my RSS feed. Thanks for visiting!

In my first article I covered the actual interview process. In this next part I now take a step back and look at how you find the most talented programmers when you have hundreds of CV’s and you don’t want to interview them all.

How to start

Once you start receiving CV’s from candidates either from directly advertising yourself or from agencies (I will be doing a post as part of this series that covers recruitment agencies.) you often find yourself weighed down with large numbers of CV which you often start culling based upon criteria that are more to-do with peoples CV presentation skill than upon their technical abilities.

Although it is easy to trim the obvious bad CV’s (ones written in crayon for instance - This has happened!) this will only get you so far (around 50-75% reduction in my experience).
After this you may still be left with more candidates than that you are happy to either telephone interview or face to face. What you need is a more reliable method that can be applied which also scales (can you tell I am a programmer?).

I present below a range of solutions that can apply to any size company.

Online Testing Companies

There are numerous online testing facilities which you can use to help filter out the less talented candidates. They mostly offer a range online tests which must be performed within certain time constraints. Sometimes these facilities are offered by the agencies that are also supplying your candidates, although I do see this is a conflict of interests as the agency wants everyone to pass.

Advantages

• Quick to get going, as you can normally choose from a range of tests (or combination of tests) on a range of subjects.
• Easy to administer. You are likely to have a company login that allows you to review results and download/export these results so you can integrate this data with your own candidate database.

Disadvantages

• The test questions are being reused many times between many companies so there is always the risk that the question/solution is available on the internet somewhere. Some companies do give you statistics on how many times they re-use each question, if they don’t then its probably worth asking.
• The process is out of your control. You are therefore relying on the system being 100% available and that any issues that come up during usage that they are dealt with efficiently by the company running the tests. This like any other service provider is a matter of due diligence when selecting the testing company.

Emailing programming tests

I have on several occasions been asked via email to perform programming tests and I have also used this method myself a couple of times. The key to making this work is to not make the task too large. I think any solution that requires that they write more than 100 lines of code is going to put them off doing it in the first place. In comparison I do not believe there are any limits too how complicated you make the task, (based upon your requirements) as even complex tasks can have solutions that fit within 100 lines of code (and putting 1000 characters on every line does not count).

Advantages

• If you already have some pre-made questions then this is a quick solution as you can email each candidate as you get them.
• The candidate gets time without any pressure to work on the problem as though they were sitting in a normal working environment.

Disadvantages

• Because the candidate is not working in a controlled environment they may decide to use an online resource to find a solution. I have had this happen several times and both were caught out at the interview because they soon showed they did not understand the solution they had given.
• Difficult to track the results unless you are very organised

Cheating
To alleviate the issues of cheating it is worth pointing out the following in your email to the candidate.

1. Specify that online research is allowed (just like you were at work.)
2. If they are successful in getting a face to face interview that they will be asked questions regarding the solution they give.
3. The solution they give will be compared with online sources for signs of cheating.

Writing your own test system

If you do not like any of the above then you probably need to write your own system. This is only really applicable to a medium-large scale company that has the spare time to design/code/test. The full details of writing a full test system are not within the scope of this article but I give here a few pointers.

Requirements

• For best effect the tests should be time limited, the timing itself must be judged by having internal developers do the tests themselves.
• Although some question/answers can be done with multiple choice (cough, guess…) it is worth implementing a system that does require humans to mark the results as well.
• Using virtualisation it is possible to produce online development environments where the candidate can work with real tools to solve a solution. This requires immense amounts of effort to setup/upkeep but has been proven to work.

Advantages

• System can be as specific to your company as you require.
• Future customisation is easier
• Good return on investment long term.

Disadvantages

• Only Applicable to medium-large scale companies.
• Costly to develop in the first instance.
• Ongoing requirement to write new tests.
• Ongoing cost to maintain the testing environment.

Last Point

My last point on upfront testing is one of applying it to the right level of programmer. Programmers can be an arrogant bunch and questioning their abilities can sometimes be a major deterrent. If you are recruiting more senior positions I would never advocate using any of the above methods as you may just find them going somewhere else. It is also the natural order of things that you are very unlikely to get 1000’s of senior candidates applying (unless your Google) so this may not ever come up.

In part 3 I will cover what questions and tests can be used within the interviewing environment and how to follow up after the interview.