This is a guest post by Albin Sunnanbo introducing a great hack to work with mails in test environments.
If you have a .NET application that sends emails, this is probably something for you.
PickupMailViewer is a simple web viewer for emails saved by the
specifiedPickupDirectory SMTP setting in a .NET application.
Download the source, publish to your test server, configure pickup directory and you should be up and running within five minutes.
Outgoing Emails in Test Environments
In your test environment will typically not send real emails, but rather use the specifiedPickupDirectory delivery method for your SMTP-settings in
web.config. This puts all outgoing emails as *.eml files in the file system instead of sending real emails.
IMHO that is the way to go regarding emails in your test environment.
However, there is one drawback, the emails gets dropped in a folder somewhere on your test server. Typically in a location that nobody looks at regularly. In my case I first have to connect a VPN, then open a remote desktop connection to our server, open the folder and copy the desired file back to my own computer (no eml viewer, a.k.a. Outlook, on the test server) and finally open it in Outlook.
Even worse for our testers that don’t even have permissions to login on the test machine. They have to ask a developer to get their emails out of the test system. As you can imagine this only happens when it is absolutely necessary.
This is a guest post by Albin Sunnanbo sharing experiences on regression testing.
On several occasions I have worked with systems that processed lots of work items with a fairly complicated algorithm. When doing a larger rewrite of such an algorithm you want to regression test your algorithm. Of course you have a bunch of unit tests and/or integration tests that maps to your requirements, but unit tests tends to test systems from one angle and you need to complement with other test methods to test it from other angles too. We have used copy of production data to run a comprehensive regression test with just a few hours of work.
Our systems had the following workflow
- Users or imports produces some kind of work item in the system, i.e. orders.
- There is a completion phase of the work where the user commits each work item and make the result final, i.e. sends the order.
- Once each item is final the system processes the work item and produces an output that is saved in a database before it is exported to another system.
We have successfully used the following approach to regression testing for those kind of algorithms.
To reach 100% testing coverage is a dream for many teams. The metric used is code coverage for tests, but is that enough? Unfortunately not. Code line coverage is not the same as functional coverage. And it is full functional coverage that really matters in the end.
Look at a simple method that formats a string.
public static string Format(int? value)
const int defaultValue = 42;
value = defaultValue;
return "The value is " + defaultValue + ".";
There is a correct test for this method, that pass and gives 100% code coverage. Still there is a severe bug in there. Can you spot it? (If you don’t, just keep reading on, it will be obvious when another test is added later.)
When developing a system that sends mails, often the mails shouldn’t be sent for real when testing. Instead they should be made available for investigation. Fortunately, that functionality is built in with the .NET SmtpClient.
There is even no need to change the code. It’s just a matter of configuration. Add the following lines to the
web.config for web applications)
<smtp deliveryMethod="SpecifiedPickupDirectory" from="email@example.com">
<specifiedPickupDirectory pickupDirectoryLocation="c:\temp" />
<!-- The network host setting isn't used, but without it an exception
occurs when disposing of the SmtpClient.-->
The pickup directory setting is meant to be used with a local mail server that watches a directory for new mails. I have no mail server watching my
c:\temp directory. Instead, the mails are just dropped there as
.eml-files that can be opened using a mail program (e.g. outlook).
In a perfect Scrum world, the team tests everything themselves. I think that misses an important point – the developers have a code-centric view on the domain. Good testing requires a user- or business-centric view on the domain. I think that it is impossible to both have a deep understanding of the code and to be a good tester.
That doesn’t relieve the developers from tests – developers having any pride in what they do of course unit test all their code. To get high quality software unit tests (whether automated or not, I’ll leave that discussion outside this post) is important, but not alone sufficient. There have to be system level tests and user/acceptance tests too.