up vote 6 down vote favorite

I could be remembering this totally wrong, but I thought there was a time when there were stacks of close votes, first posts and late answers awaiting review. Then one day, when I noticed the review tool changed and a few new badges appearing, all the counts dropped to zero within a few days, where they seem to remain, with minor perturbations. (As an aside: is it just me or did the question count jump by 40k in just two or three months?)

I like the auto-queue format, but it makes me wonder about the quality of the review that's going on. It looks like limits were imposed to discourage too much streak reviewing, but now it looks like hardly any users are able to perform more than a handful of any given review type per day anyhow.

How can we adjust the system to balance rapid review with quality consideration?

EDIT: Here is one suggestion: How about bumping up the number of reviews needed to confirm an edit slightly? (I'm not even sure what it is now, nor am I sure what the average lifespan of a pending edit is.)

share
The question count by month can be found here. The visible speed of review depends on the number of users doing it more than on the time an individual user spends pondering each item. So I do not take the perceived quickness as evidence of rash decisions. Is there any other evidence? – user53153 Dec 31 '12 at 17:08
@PavelM Thanks for the link! I don't have any hard data, just the anecdotal evidence I have seen where a majority of reviewers make an obvoiusly bad call on a review. I would expect that if everyone is spending sufficient time, these bad reviews would be more rare than that. – rschwieb Dec 31 '12 at 18:02
2  
It would be nice if some meta-SE guru gave us the current status of review queue policies. I understand they've been tweaked a few times, and I'm not sure which of the meta threads have current information. – user53153 Dec 31 '12 at 20:45
4  
It's a common fallacy that two people glancing at an item will have better judgement than a single person feeling solely responsible for the outcome. – Phira Jan 2 at 0:15
@Phira It's like in statistics: if you want a better estimate, you can increase your sample size. I think calling it a fallacy is unfounded (although if your point is that a sample size of two is too small, then I can agree.) While these calls are very subjective measurements, it seems likely that having more people looking at it would generate better readings. – rschwieb Jan 2 at 14:29
1  
Condorcet's Jury Theorem, perhaps? – Charles Jan 2 at 23:33
3  
I just found that StackOverflow is watching the watchmen by automatically planting low-quality items in the review queue. Reviewers can get suspended from reviewing if they do not pass the test. What is the world coming to... – user53153 Jan 3 at 4:07
No, please don't increase the number of reviews needed to confirm an edit even further -- I already think it's a bad idea that it requires more than one. – joriki Jan 7 at 10:03
@joriki The opinion is a little useful but I would also like to hear your reason! Thanks... – rschwieb Jan 7 at 14:19

You must log in to answer this question.

Browse other questions tagged