Finalists can be selected by community vote, jury selection or a combination of both. Most crowdsourcing platforms offer features that allow you to filter ideas via votes received, among other criteria, making it easier to organise. While the highest-voted ideas are not guaranteed to be the strongest, they are a good indicator of ideas the community will support to actualize.
It’s worth noting that crowdfunding platforms like Kickstarter and Spacehive have seen ideas that receive 10% of their funding target in the early stages of their campaign have a 79% greater chance of coming into fruition through community support.
a) Community selection
It’s a powerful gesture to allow the community to have a say on the finalists. If you decide to use online voting to select the winning idea, you’ll need to decide whether you’ll have a 100% community vote, use the votes to shortlist and then allow an expert panel to select the finalist, or do a mixture of both. You may even want to have an additional category for the “community choice” award.
How you handle this also depends on whether you have a multi-stage selection process. When it comes to funding projects and paying more than $5K or so, it’s important to see the idea-submitters pitch to get an impression of their level of passion and determination. When “crowdsourcing for impact”, we therefore recommend to have a selection committee that makes a final decision.
Options include:
It's important to give the community a chance to make a difference, and therefore essential to involve them in the selection of the winners (if they have no say on the finalists, they might not be motivated to show support by voting). This can easily be achieved by online voting, although some people have more friends than others, thus artificially shaping the vote. A good option, therefore, is to let the community make a pre-selection, and to have a jury make the final decision, based on pitches. When selecting the finalists, the jury should take into account the result of the online voting, but it should be free to differ slightly.
Although it’s good for the community to have a say, we wouldn't recommend a 100% community vote for the pre-selection, because it’s possible that some of those selected will have no chance of being chosen by the panel – and by asking them to pitch, you’re wasting the time of both the participants and the panel. The results, in this case, are often highly biased, so members may see the process as meaningless, therefore losing their trust.
It’s good to let the community vote because – most of the time – the top-voted ideas are really good. If you select 10 of these, the community and the experts often agree on seven or so. The experts may wish to stick closely to the voting results, thus empowering the community and aligning with their views. Most people know that blindly voting for their friends cannot save a poor project, so tend to spend less time doing that, resulting in less bias in the results.
You can also mitigate popularity bias by not guaranteeing spots, instead saying that the selection committee will pay close attention to the top voted ideas.
We have tested the early-bird method (also called “sprint challenges”), and although they do introduce some popularity bias, it can be a great strategy to get people active at an early stage. It doesn’t really matter if one or two out of ten got there because of their large networks, because they may have been the most popular in the end.
b) Online voting
Thumbs-up
Using a thumbs-up icon to register votes can give you a clearer indication of popularity. But for closed communities, it can be helpful to limit the number of votes so that you can see who the clear “winner” is. For example, if you’re you looking for ten finalists, limit the number of votes a participant can cast to ten, so that you don't end up with lots of ideas with the same number of votes. The other benefit of the thumbs-up option is that many participants will be familiar with it, because a lot of platforms (including Facebook) use this.
We’d suggest disabling the thumbs-down option, as it can be abused and is generally not very helpful, because you need to know why someone doesn't like an idea. Instead, a participant should leave a comment with “constructive feedback”. See here for more on community culture.
Stars
By introducing stars, participants are asked to think beyond just liking an idea and assess for quality, which can be a more interesting process for them. However, statistically, this will not make a difference if you have a sufficiently large number of participants – ie, you will get the same relative ranking. Most people tend to choose the higher numbers on a 5-star scale – most people give 4 or 5 stars, and 1 or 2 are rarely used. Our recommendation is therefore to limit the number of stars to 3, in order to make sure stars are quite evenly spread. It’s also helpful to add a little guidance for participants on the weighting of the stars, such as:
Not good/no opinion – 0 star
Needs work – 1 star
Good – 2 stars
Great – 3 stars
One important decision needs to be made at this point: are you using a system where stars are added as points (as in, no star equals 0 and therefore gets voted down) or where you calculate an average (as in, no vote is harmless, keeping the idea at the previous average)?
If you choose for "0" votes to count as a down-vote, then you end up with a similar problem to the thumbs-down option, where it’s unhelpful and can be abused. Therefore, we’d recommend that having no stars doesn’t affect the placement of the idea.
Blind voting vs. open voting
In our experience, you get more objective results in internal challenges if you set up “blind voting”; this means that nobody knows how many points each idea has. This is the opposite of open voting, where voting figures tends to be an incentive for participants to spread the news about the challenge. It’s worth noting that if you use stars or rating scales with open voting, most people award higher ratings.
Inviting friends to vote
When voting is visible and participants are allowed to invite friends to vote, it might be a great opportunity to grow the community quickly, but it can also be a risk to community health.
It’s important to allow participants to invite others to vote, as it’s a good test of their marketing skills, as well as their ability to spread the word and build support around their idea. But the moderator should try and respond to some of the voters and ask them to leave a helpful comment under the idea, too (avoiding generic messages of support, such as “good luck!”).
If you're worried about the challenge turning into a popularity contest, you can set a restriction on the number of people each participant can invite to the platform.
Click here for further advice on counting votes and communicating the winners.
