dot.net.nz has got it good!
Andy Oram posted to the O'Reilly Radar about his recent research on mailing lists, and linked to his article
"How to Help Mailing Lists Help Readers".
In his article, Andy followed threads on some mailing lists (Linux, Perl, Ruby), and uncovered some patterns of behaviour (summarised below):
- Many questions aren't satisfactorily answered (46%)
- Helpers give up after a few attempts
- Beginner users have fundamental gaps in knowledge, and need direction to other documentation sources
Now despite recent unrest about the effectiveness of the dotnet mailing list (at www.dot.net.nz), in particular the performance of the mail sender, I've always been convinced of the relevance of the answers given on the list, and impressed by the tone of the replies. We've got a nice little community going, and people offer quite in-depth help wherever they can.
I'm always impressed when people go out of their way to solve a problem, such as installing a piece of software to help diagnose someones problem (
Ivan Towlson, I think that was you :) ), or reliably pitching in to solve a problem (PeterB and "shane ~" are among the regular "helpers").
So I collected some statistics following a similar process to Andy's:
- 15 recent threads from the NZ dotnet mailing list where a specific question was asked
- I measured similar statistics on the effectiveness and time to resolution, although for some of the threads where the orginal poster didn't reply with thanks, I defined "resolution" subjectively as whether I thought a satisfactory answer had been given
- A count of the number of messages in each thread, the number of helpful / on topic messages, off-topic, irrelevant and unhelpful messages
Results:
- 80% of questions received a satisfactory reply!
- There were no off-topic, irrelevant or unhelpful messages!
- Median time to resolution was 20 minutes!
- Longest time to resolution was 2hrs 34mins.
- Best response times are early-mid morning, and mid-afternoon. Slower responses over lunch time.
I expected to see the dotnet list coming out well, but when you compare these numbers to the ones Andy collected, and even if you factor in some differences due to sampling / processing technique, the differences are staggering:
Table 1 (modified). Resolution times for questions on mailing lists
|
Minimum |
Median |
Maximum |
| NZ .NET list |
8 mins |
20 mins |
2 hours, 34 mins |
| Perl |
2
hours |
8
hours |
1
day, 21 hours |
| Rails |
0.5
hours |
16
hours |
7
days, 10 hours |
| Both operating
systems |
0.1
hours |
10.5
hours |
2
days, 10 hours |
| Both languages |
0.5
hours |
13.5
hours |
7
days, 10 hours |
| All lists |
0.1
hours |
11.5
hours |
7
days, 10 hours |
Some of the things that I believe make the dotnet list so successful:
- The list is an extension of the user groups, and many people know each other in person
- Everyone is in the same timezone and industry, which means the responses are fast, and usually at the same time of day that you need help
- Off-topic conversation is kept to a seperate off-topic mailing list
- The list subscribers have a wide spread of knowledge and experience, with some members having very deep .NET knowledge
- Timaru is discussed monthly :)
The list server that runs the mailing lists (dotnet, sqlserver, dotnet-offtopic and others) has been tirelessly maintained by
Lukas Svoboda over the past 5 or 6 years, and the not inconsiderable costs of sending out many thousands of emails a day has been sponsored by him, Microsoft, Irongate, Orbiz, Intergen and others over the years. Thanks!