Monday, December 21, 2009

Poll Result & Analysis—Bolded Questions

For about 2 weeks up to yesterday, I conducted a poll on my sidebar. It's regarding what's known as the "bolded questions" which sometimes appear in treasure hunts. They are basically the same as the average hunt questions except that they're much harder to solve.

The bolded questions are a common feature of some hunts, e.g. the Kiwanis Treasure Hunts in West Malaysia. The first time I encountered those bolded questions, I found it a bit strange that the points allocated for those questions were much lesser than those allocated for the average hunt questions which were easier to solve. The explanation I got from the regular hunters was that the bolded questions were designed to "level the playing field" by tempting the stronger hunters to spend too much time on some tougher questions at the expense of losing the time to solve the other easier questions.

So in the end the strong hunters may find that they're able to solve the tougher questions and gain a few points from them, but losing the opportunity to solve the easier questions because of insufficient time! And because the easier questions carry more points, the regular hunters may end up with a lower overall score against the weaker hunters.

However, I have also noticed that many strong hunters no longer fall for the trick. Many of the strong hunters are strong not only in terms of solving questions and spotting answers, but also in terms of time management. Having been a passenger in a strong team, I was able to note that the master hunters would actually forgo the bolded questions after only a short time trying to solve them! They did so because they felt that it was not worth the time investment for the small return in terms of score. In fact, they'd rather take their chances with the other route questions.

In at least one of my hunts, I experimented with the bolded questions and allocated lower points for them. I was somewhat disappointed that over 90% of the teams did not even bother about them at all!

So what usually happens is that in spite of the extra time and efforts by the setter when setting the bolded questions, they would end up not even attempted by the hunters! And that is truly a shame; I would consider that a waste! In my opinion, if more points are allocated for the bolded questions, maybe the stronger teams would be more willing to gamble on their time in return for a, hopefully, higher score in the end.

Recently, I asked some master hunters about the bolded questions. And a few of them responded that they're big fans of more points for bolded questions, not the other way round. I must say that I'm inclined to agree with them. So I decided to conduct a poll on my sidebar.

I gave 4 choices about how successful answers to bolded questions should be rewarded:

1) Lesser points than the average questions

2) Same points as the average questions

3) More points than the average questions

4) Whichever way the setter decides

Only 28 people responded to the poll, and I suspect most of them were from KK.

It is interesting that no one opted for (1), although that's the norm of hunts where bolded questions are available.

6 people or 21% voted for (2), thus indicating that that is a better way to be fair to both the strong and weak hunters. There is no extra incentive to work harder to solve the tougher questions. It means that the stronger hunter may have to spend say half an hour on a single bolded question but risk losing an easy question (because of insufficient time) and end up on par with the weaker hunter!

15 people or 53% voted for (3), suggesting that more points for tougher questions is still the most popular choice. In my opinion, it is also a logical and fair choice. Striving hard to crack a tough riddle in return for a miserable reward has never really been agreeable to me anyway!

7 people or 25% voted for (4), suggesting that they're willing to let the setter call the shots. These are probably those who challenge themselves to survive in whatever jungle they're thrown into.

However, I feel that 28 people may be too small a sample to reflect the opinions of the entire hunting fraternity.

To the KK hunters, I think you can probably guess that you'll be seeing some bolded questions in my KK Challenge 6; and the kind of system I'll be adopting for the scoring.

18 comments:

WK Azlan said...

Oh gosh...I can't believe I actually missed out on the poll, but you've made some valid arguments there...for me personally I would definitely choose (3) "More points than the average questions", as that would entice me to try out the more difficult/mind boggling/headache inducing/"masters" questions...just for the fun of it! :)

Claire said...

Quite a shame ain't it if the "bold" questions were left unattended when I'm sure a lot of hard work by the CoC has been poured into creating these questions!

Cornelius said...

Yes, Azlan, I suspect even the average and new hunters would prefer option (3).

I believe it would be much harder for the master hunters to simply ignore the bolded questions if they're worth a lot of points. They will be afraid that other masters can solve and thus earn those extra points. So I think they will be "forced" to spend at least a bit of extra time to solve. But this can be beneficial for the weaker teams when and if the master hunters end up ruining their time management.

Cornelius said...

Tell me about it, Claire. It's quite a shame and very disappointing. The time and effort I spent to set the questions, and then seeing them ignored!... arghh!

teo said...

Hi Corny,

Why not keep the "Bold" Questions UNbold so that your hard work would not be ignored. At least without the Bold, all hunters will try to attempt without any fear!! Let the "Bold" questions hide between the normal ones ..... just my half cent.

BTW, I heard from Mary just now in the gym that ISM Hunt will be postponed to March? So KK Challenge should replace ISM Hunt in Jan, KK City Hunt so far very quiet......seems like 2010 will be another quiet year for the hunters.

Cornelius said...

teo,

I think better to make it clear that the question is tougher. But tempting the hunters with higher score. I'm just gonna experiment to see if there is any difference with the higher score. I think more people will try to solve if higher reward. But we will see how it goes.

Yes, teo, unfortunately I received an SMS from Bernard recently. And he said he's postponing the ISM hunt to March.

It's not really a big problem for me to set my KK Challenge hunt, but I don't think it's a very good idea to do it in January. Better to spread it out a bit. Since we're having the KK City hunt in February (by TOS again), and ISM in March, I should stick to my usual July/August for KK Challenge 6. I don't know if I can "force" Alvin to do one in May. If he can, that would be great too.

Anonymous said...

I remember a few years back when I took part in Mensa Hunt. They have so call tie breaker questions. No point for correct answer but it is only useful when there is a tie break. The question not only tough but also not in sequence. For me the normal one already put my team in trouble so we just ignore this kind of questions in order to save our time. But Strong hunters still put their effort to answer it and some of them get all correct! So I think if lesser point will be given for bolded questions and end up left unattended,it is not just a wasted but also normal hunters will loose oportunity to learn how to crack tough questions.

Cornelius said...

Ah! yes, that is also another possible way to look at it. That's why it is very strange that bolded questions are frequently worth lesser points (or in your example, no points at all) than the average questions. There is very little incentive to try to crack them!

2 Romans 1 Impostor said...

I beg to differ. Unless one is shooting to finish outside the medals, every single point is precious. So you can bet your bottom dollar, that the competitive teams do attempt all questions even those that are allocated lesser points.

Cornelius said...

Oh yes, 2R1I, of course the teams will attempt all the questions. Even the weak teams will do that. But not all of them will "seriously attempt". They merely spend a short moment and then if still nothing, that's it!

I had at least one proof of this when hunting together with a strong master, and we all adopted this strategy. We tried, and then after a short while, we said something like, "OK, let's move on, this is just worth 1 point. Better to focus on the other questions."

That's why now I want to experiment to see if there's any difference if more points are allocated for the bolded questions.

renroc, would you like to add anything on this?

renroc said...

Most teams, strong or weak, will like to attempt all questions and will probably do so as long as they are not in time-trouble. Likewise, for most teams (regardless of hunting strength), the low-value bolded Qs will be the first to be dropped when they are in time trouble. The main difference is that the stronger teams tend to reach a higher degree of time trouble before they are willing to consider dropping Qs.

Cornelius said...

Ah! you put it more elegantly than me, renroc. Yes, that's what I mean.

However, I'd also like to comment a bit further about fighting for even a single point. I can agree with 2R1I as far as that is concerned. I can make the comparison between the 2 times I hunted in the Kiwanis Hunt. Once was with renroc and once with HRU.

In the case of renroc and his friends, we did try to solve the bolded Qs, but we were not so reluctant to drop them when we were unable to solve.

In the case of HRU, there was a bit of a "split decision." I can remember KK Chai wanting to drop some tough Qs after we have spent what I felt was too much time on them - decision which I thought was the correct one. The rationale was that we could save the time to tackle other questions ahead. But VK was of the opinion that "if we kept dropping Qs, we would have no chance for the win no matter what happens ahead, because we would not have the opportunity to come back even if we had those extra minutes. (It was an outstation hunt)

As I said, I'm inclined to agree with KK Chai. However, VK had a good point too. It's true that once we've dropped some Qs during the earlier stage of the hunt, we wouldn't have had the opportunity to pick them up again later even if we had the extra minutes. It turned out that for that particular hunt, we actually ran into serious time trouble when we reached the last sector in Johor. And because we were working under serious time trouble, I felt we failed in at least 4 - 5 Qs which I'd like to think we could've solved if we had the average time for them.

But this is all seeing the event in hindsight. Looking back at that hunt from where I am right now, I believe we could've improved our position by quite a few rungs, but still not win the hunt.

I think it all boils down to some kind of balance between KK Chai's and VK's arguments, both of which are valid arguments. In the game of chess, we sometimes get into the same situation. When we absolutely have to win, we have the tendency to "risk it all," and may end up with a worse outcome.

Some master hunters, although trying very hard to fight for every single point, have the perfect balance to drop those Qs when time no longer justifies them to keep trying. But I think if the points allocated for those Qs are more, maybe greed can make them linger on a "little" longer! And that would be interesting to see!

Cornelius said...

Oops! I meant to say GREED and THE FEAR OF OTHER TEAMS SOLVING THOSE QUESTIONS might make them linger on a little longer.

2 Romans 1 Impostor said...

Ah! The 3,000 dollar question: To drop or not to drop (Questions)?

Truly, there is no Golden response.

In certain hunts, dropping 10 Qs is still good enough to win the hunt while in other hunts, a perfect score may not even guarantee a finish amongst the medals.

The decision I would say is instinctive, which one tries very hard to piece into the team strategy and convince ones team mates :).

Factors that can influence one's decision would be (and not limited to):

a) How ambitious is one?
Solving the bolded questions most of the time gives one's team an edge. If one is ambitious to finish at the top of the ladder, there is no alternative but to net one or more of these bolded ones. Otherwise, expect just an average finish.

b) Level of confidence
In hunts, there will be a variety of TOUGH questions: Impossible ones - every word reads like a foreign word; Imposing ones - tough but with a light at the end of the tunnel (sufficient clues that make enough sense). My advice is to JUST TAKE OFF AND GO on the Impossible ones, and spend the time on the Imposing ones where one at least has half a chance of finding the solution.

c) Level of difficulty of hunt
Teams that plan their questions well can grossly estimate the level of difficulty of the overall hunt. One may have the luxury to spend more time on the tougher questions in the early part of the hunt, if the hunt is bottom-easy.

d) CoC's reputation
Try as they might, some CoCs just cannot craft challenging questions (to today's standards). In their hunts, you instinctively know that dropping one question would be disasterous.

Whether 1 point or 3 points, to me every point is precious, more so the bolded ones! Cheers.

Cornelius said...

Granted, 2R1I, I could see your attitude when we hunted together. And I'm not saying that's a wrong attitude! In fact, most of the time I am like that too!

And I can also agree on your points (a) to (d). There is no question that every point counts! I am not disputing that at all. But somewhere in the hunt there's the element of gambling. One gambles one's time in the hope of solving a seemingly solvable bolded Q in return for a miserable point. And maybe after 30 minutes, one is able to solve. That will be very good. But there is also that possibility that one gets nothing after 45 minutes. That will be a shame.

But there is also the strategy that after spending 15 minutes (depending on how much more time is available), one gives up on that bolded Q and use the "extra" 15 minutes to solve an average Q instead. It makes sense because each average Q earns 3 points, whereas a bolded Q earns only 1 point. There is no guarantee that investing 30 minutes on a bolded Q will yield the 1 point, but there is that very promising 3 points awaiting for the average Q. That's why I said it's a balancing act.

We can't say which is the superior strategy because there is the element of luck involved. And also as you have correctly pointed out those elements in (a) to (d).

But this poll is not really about an individual strategy. I'm just trying to find out whether there is any difference at all if a higher point allocation is allowed for bolded Qs. Clearly, in your case, there is no difference at all. Your point is, you will try to answer ALL the Qs, 1 point or 3 points. But I think others may not share the same attitude. I think others may do a bit of weighing the pros and cons of the points, i.e. whether it's worth it to invest such precious minutes for 1 point only when a single average Q can earn 3 points. Because for every single minute spent on the bolded Q, one will jeopardize the chances of solving the average Q.

I think if I allocate say 5 points for bolded Qs against only 3 points for average Qs, then more people will be willing to invest more time on the bolded Qs, because in that case it's just too much to ignore. I'm saying this may not have any impact on you; but I think it can affect other hunters.

Cornelius said...

I'd like to add that from my observation when hunting with renroc (correct me if I'm wrong, renroc), his attitude was that it wasn't worth the heavy time investment on bolded Qs in return for 1 point each. I thought he was more willing to take his chances on the average Qs. Maybe he was hoping that the other hunters would spend too much time on the bolded Qs to the extent of losing some of the average Qs. In fact, this was what actually happened to us the last time we hunted together, 2R1I? (although in that case, we weren't really dealing with bolded Qs, just very tough ones)

I have a feeling that renroc may think differently if more points are allocated for bolded Qs. I think he would try much harder to solve them, and perhaps he'd be more willing to invest a bit more time on them! What say you, renroc?

Cornelius said...

I was exchanging Christmas greetings with a hunter friend this morning, and then we got to exchange opinions about this post and the comments in it. I found it strange that he said that I appeared to blame my team mates for not doing well in the Kiwanis Hunt!

I want to say here that that is not true. If there is any regret at all, it must be the fact that I did not have it in me to contribute very much to HRU. The team members of HRU are very good in what they do - there is no doubt that they are a very well-oiled hunting machine.

I am a strong believer of teamwork. We win or lose together. We may debate on some decisions, but once those decisions are made, they are collective decisions. So there is no blame or credit to a single person. We are all responsible for whatever decisions we make as a team, good or bad.

I also want to say here that of the very few hunts that I joined this year, I found that, on the whole, the Kiwanis Hunt was the best. I think Wei Ming did a good job. People always come to the wrong conclusion when I comment negatively on some questions in hunts. When taken as a whole, I thought the questions in the Kiwanis hunt were just the correct blend. The winning team failed in only 2 or 3 Qs. That is quite an achievement for both the hunters and the CoC in my opinion.

renroc said...

There's nothing for me to correct. :)

Agree with you too that whichever strategy or attitude will not be right or wrong all the time - it is always a gamble and depends on the many factors that you & 2R+1I have discussed.