Programmer Testing

One issue about hiring programmers is how to test their technical competencies. If you merely look at their CV, that is only part of the story. I would like to propose a simple way to actually test their ability to problem solve by writing a program.

There are many on-line testing sites available. However, these are all language dependent. Therefore, a programmer’s experience in the language does make a difference. Hiring for a narrow language is a bad idea and it’s better to test on general logic capabilities.

This is where I think that something like the LLVM intermediate language is useful. It is unlikely that anyone has ever learned to use that language before as it is exclusively generated by the compiler. Therefore, this eliminates any language bias that may inadvertently affect the candidates.

To be fair, I think that the candidate should be assigned the tasks and given a suitable time frame to learn the language and complete the task as necessary. I believe that a one-week time-frame should be sufficient to learn enough of the language to write simple programs. This also tests the ability of the candidate to pick up new languages, which is a necessity in our line.

It is also a powerful enough language to do anything. So, the programming tests can be designed to be as easy or as complicated as necessary.

Furthermore, it is an actual programming language that can be compiled into real machine code. This allows the written programs to be tested for functionality as well as performance. It is possible to throw a barrage of test cases at the generated application to see how it performs. The entire system can even be run in a sandbox.

The only thing that is needed then, is an interface to the system. Technically speaking, the candidates can use whatever text editor that they want, and they would only need to upload the code to a centralised testing server, which would then compile and test the code.

For testing digital design skills, an equivalent method can be drawn from using a dead meta language like Confluent. It is, again, highly unlikely that any person has had much experience with the language. The output can be used to generate syntactically correct Verilog/VHDL and tested with standard tools.

I think that this would be a cool project to work on, if I only had the time. Maybe it’s time to hire another intern to do it.


Published by

Shawn Tan

Chip Doctor, Chartered Engineer, Entrepreneur, Law Graduate.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s