Thoughts on POLLS

[pandaman]

There have been a few polls on here recently, and this is not one of them, just a few thoughts and observations on forum polls in general (not just the recent ones).

1.  Obviously no one can, or should, (or does afaik) claim that any polls on here are representative of the player base in general.  So as a way of gathering representative data of the general player base they are pretty much useless.

2.  Given that any poll on here will only be representative of those forum users that actually vote on it they can still give some worthwhile data and offer a useful insight into the questions raised, depending on what they are. So polls can be of use and/or interest.

3.  A big problem often seems to be that the creators do not often explain very well what they are trying to find out (and yes, English is not the first language for many, so naturally this side of it can be much more difficult for some).  A more detailed 'OP' outlining clearly what the poll is for, what 'data' they want to collect and why could help enormously for those voting, and therefore should give better data back. 

4. Often the voting options can be unclear or confusing, sometimes appear weighted to suit the OPs opinion (though maybe not deliberately), or simply too complex or not comprehensive enough. 

Ok.  My conclusion (for what it's worth)...  Forum polls can be  interesting, fun, and potentially useful if done well, so there is a definite a place for them.  However, thought must be given to exactly what you want to find out before writing the questions/options and then word them clearly accordingly.  Importantly a good OP that explains the purpose of the poll, and what data you hope to collect, will ensure that voters are in a better position to vote properly and ask any questions should they need to clarify things. 

A good poll can generate interest and therfore stand a chance of getting the data you want.  A poorly worded one will just put people off and generate negative responses.

[regex]

2 minutes ago, pandaman said:

3.  A big problem often seems to be that the creators do not often explain very well what they are trying to find out

That's often because they're trying to engender a bias in the poll-takers towards an outcome that backs up their own conclusions to the poll.

[Rainbowtrout]

Moved over to the Kerbal Network, where forum features and other such things are discussed.

On-topic, I agree that polls can be easily biased towards the creator's preferences. But, you can't expect the OP of a poll to scrutinise the question and the options to make it evenly balanced. If they did that, then polls would be a lot more accurate. The real question is, why didn't you create a poll to discuss polls?

[DChurchill]

12 minutes ago, Rainbowtrout said:

Moved over to the Kerbal Network, where forum features and other such things are discussed.

On-topic, I agree that polls can be easily biased towards the creator's preferences. But, you can't expect the OP of a poll to scrutinise the question and the options to make it evenly balanced. If they did that, then polls would be a lot more accurate. The real question is, why didn't you create a poll to discuss polls?

Because then you'd have a poll to poll paradox which would tear space and time asunder!!!!!!!11!!11!!Eleven

[Superfluous J]

Having made my own poll recently, I'm kind of of the mind now that they don't actually garner any useful information whatsoever. At the absolute best, they can only really tell you the opinions of those who play the game, visit the forum, see the poll, care enough to vote, and answer truthfully. That is such a skewed sample base that I don't know of any way that any of the information can be considered useful at all, including the ability to offer insight.

I mean, sure, that could happen. But it could also happen on a non-poll thread. I see no reason to think it's more likely to happen in a poll than not.

[The_Rocketeer]

26 minutes ago, regex said:

That's often because they're trying to engender a bias in the poll-takers towards an outcome that backs up their own conclusions to the poll.

Speaking as one who has created 2 recent polls this is categorically not true of me. Bias is often inevitable, but that doesn't make it deliberate.

 

34 minutes ago, pandaman said:

2.  Given that any poll on here will only be representative of those forum users that actually vote on it they can still give some worthwhile data and offer a useful insight into the questions raised, depending on what they are. So polls can be of use and/or interest.

3.  A big problem often seems to be that the creators do not often explain very well what they are trying to find out (and yes, English is not the first language for many, so naturally this side of it can be much more difficult for some).  A more detailed 'OP' outlining clearly what the poll is for, what 'data' they want to collect and why could help enormously for those voting, and therefore should give better data back. 

4. Often the voting options can be unclear or confusing, sometimes appear weighted to suit the OPs opinion (though maybe not deliberately), or simply too complex or not comprehensive enough. 

Ok.  My conclusion (for what it's worth)...  Forum polls can be  interesting, fun, and potentially useful if done well, so there is a definite a place for them.  However, thought must be given to exactly what you want to find out before writing the questions/options and then word them clearly accordingly.  Importantly a good OP that explains the purpose of the poll, and what data you hope to collect, will ensure that voters are in a better position to vote properly and ask any questions should they need to clarify things. 

A good poll can generate interest and therfore stand a chance of getting the data you want.  A poorly worded one will just put people off and generate negative responses.

Point 2, I think 'interest' is more to the point than 'useful'. Whether they are useful implies that they have a purpose beyond being interesting.

Point 3, as pollsters, I don't think that what you're trying to find out is relevant. If anything this will cause an even deeper level of skew. Better by far to keep the responders blind to your own views beyond the words in the question.

Point 4, well you've got to set some limits otherwise polls are so long and scathing that they feel like an inquisition. They need to be concise and brief. Bias not so good, but even biased polls aren't really a problem because:

If you dislike, disagree with or generally disapprove of any poll you are totally at liberty to to ignore it. Nobody should feel that creating a poll for whatever reason isn't an option for them. Polls are an interesting and different way to collect views without getting everybody to write a reply and then tallying up who thinks what. Personally I'm done polling for now because I'm sick of the bad taste it leaves in my mouth when all I'm doing is asking everybody a question. This is what it comes down to - I'm interested in what answers I might get to a question, but I don't want a long drawn out thread about it that's bound to be heated, sometimes offensive, and end up spinning off on wild tangents. Polls keep the question to the point. That's all I have to say about that.

[regex]

14 minutes ago, The_Rocketeer said:

Speaking as one who has created 2 recent polls this is categorically not true of me.

Okay.

[pandaman]

I suppose 'polls' can come in two basic varieties. 

Gauging opinions... (Do you like X?  do you think Squad should do Y?).

And

Getting information...  (How many mods do you run?   Do you use a mouse or joystick?)

The first type often seems to 'rattle cages' and cause more 'heated' discussions l, but I guess the latter type would be far less subjective and therefore be a potential source of comparative data.

[The_Rocketeer]

I think there are a couple of fallacies misconceptions here @pandaman.

Imagine the number of people who might (for example) check a poll response that says 'KSP is unplayable because of the latest update'. Now imagine what the thread would look like if there were no poll and each of those people, rather than simply ticking a checkbox and submitting their position to the count, decided to start their own thread about how unplayable the latest patch was. I genuinely think that if the first person to notice significant problems with KSP's recent updates had started a poll rather than posting a complaint thread, there would be a much tidier, less confrontational forum. The poll serves to concentrate the topic into one single thread that addresses the extent of the problem directly through collective contribution, and how extensive the problems are is one of those areas where threads about personal experience very often tend to go off the rails (e.g. 'you're one of only a very few people saying this so you must be wrong' vs 'it seems a lot of people are having the same problem, so I sympathise')

Secondly, forum polls are all walk-up polls which are a very unscientific way of conducting any survey. Therefore the information you collect isn't very useful for comparison with any other data, unless you're comparing one forum's respondants with another forum on the same or a related subject, answering an identical poll. They could only be more scientific if they were targetted somehow, but on a public forum there is no way to enforce or ensure that your targets actually respond. So, the idea that the information can be used to draw meaningful conclusions is a bit far-fetched, which is why I have generally avoided doing so. Nonetheless, they do constitute a kind of abstract evidence that can help moderate attitudes and quasi-quantify a general mood into something a very little more solid than a vague pervasive impression. Moderation is exactly the opposite of cage-rattling. If the discussions are heated, it is probably more to do with the sensitivity to the topic than the existence of the poll - people would just get heated about it in other threads instead of 'here'.

[Guest]

5 hours ago, regex said:

That's often because they're trying to engender a bias in the poll-takers towards an outcome that backs up their own conclusions to the poll.

Maybe we should make a poll about it.

[pandaman]

8 hours ago, The_Rocketeer said:

I think there are a couple of fallacies misconceptions here @pandaman.

Imagine the number of people who might (for example) check a poll response that says 'KSP is unplayable because of the latest update'. Now imagine what the thread would look like if there were no poll and each of those people, rather than simply ticking a checkbox and submitting their position to the count, decided to start their own thread about how unplayable the latest patch was. I genuinely think that if the first person to notice significant problems with KSP's recent updates had started a poll rather than posting a complaint thread, there would be a much tidier, less confrontational forum. The poll serves to concentrate the topic into one single thread that addresses the extent of the problem directly through collective contribution, and how extensive the problems are is one of those areas where threads about personal experience very often tend to go off the rails (e.g. 'you're one of only a very few people saying this so you must be wrong' vs 'it seems a lot of people are having the same problem, so I sympathise')

Secondly, forum polls are all walk-up polls which are a very unscientific way of conducting any survey. Therefore the information you collect isn't very useful for comparison with any other data, unless you're comparing one forum's respondants with another forum on the same or a related subject, answering an identical poll. They could only be more scientific if they were targetted somehow, but on a public forum there is no way to enforce or ensure that your targets actually respond. So, the idea that the information can be used to draw meaningful conclusions is a bit far-fetched, which is why I have generally avoided doing so. Nonetheless, they do constitute a kind of abstract evidence that can help moderate attitudes and quasi-quantify a general mood into something a very little more solid than a vague pervasive impression. Moderation is exactly the opposite of cage-rattling. If the discussions are heated, it is probably more to do with the sensitivity to the topic than the existence of the poll - people would just get heated about it in other threads instead of 'here'.

I don't disagree with you here.

My thoughts were aimed really at whatever polls that may be created, and wanting to make them a bit 'clearer' in their aims. How they are presented and worded can obviously make a big difference to that.

[The_Rocketeer]

@pandaman to me it's obvious that the aim of any poll is to seek answers to the questions they ask. If a person creates a poll with a different aim, they're using the wrong tool for the job, because that's the only thing that polls can really do - collect answers.