
Today we rolled out "Positive Feedback" rankings for users who have submitted a minimum amount of content to Newsvine. All users start out at "N/A", and after they have a base of articles, seeds, or comments, a "Positive Feedback" score will then appear next to their avatar.
This is a system which will probably get tweaked a few times before it's perfect, but it's designed to represent your positive contributions to the community versus any times your content has been reported. Some of the best writers here can be quite controversial so striving for 100% is not exactly necessary, but in going through the scores of most of the people around here, we've noticed that most people are above 70% or so, so that's good.
Since this system is so subjective, we're intensely interested in how you feel about it. Do you like seeing a percentage value up there? Would it be more useful to simple see a "star" rating system where users have between 1 and 5 stars? Currently we have set the "floor" at 50%, so anybody under 50% will just show up as "< 50%". Is this the best way to handle marginal contributors?
As always, we're open to suggestions, and we expect this to be the first iteration of something which will improve over time as we see how people react to it.
I would vote for or recommend the use of the star rating. It is more visual, but also its less numerical. That statement probably sounds funny, but consider this. I have a 100% rating (before this comment). Since I haven't posted anything, and have only successfully seeded two articles (the rest were all dupes), I'd assume my rating is mostly based on my comments. Now I admitted to commenting a lot, and I try to be reasonable, rational, and well supported (if not necessarily right).
While I'm grateful that none of my comments have been tagged as flamebait, otherwise tagged as offensive, or just marked down because someone didn't like it. I have to admit my first thought was, 'I'm ranked at a higher percentage than theannalog, Jason Coleman, kevinb66, and Aaron Ross Powell???'
Numerical values lead to mathematical comparisons; and while the overachiever in me is still standing up do a little cheer that I got a 100% (I feel like I just aced a test I didn't know about); the social justice/social responsibility activist in me says, 'that's just wrong'. Really, that is just wrong. It's not that I think my comments are crap, because I do put some effort in them, and I occasionally take not so popular positions, but I've yet to place myself in the same arena as theannalog, Aaron Ross Powell, or a number of others, because I don't post.
I think the conversation starters of the group deserve record props, because they place themselves in the position to be judge critically. The become "the liberal lassies," "the listless libertarians," and the "long-in-the-tooth conservatives". I'm certain that these individuals suffered a loss of percentage points purely based on the fact that someone didn't agree with their posts, and mostly tied their posts to their political point of view. Now, I'm not saying that's all bad; because a number of posts tie their authors to a political viewpoint, and political viewpoints will be judged. I'm saying that I shouldn't be judged higher than them, just because my political viewpoint is less well known.
That said, I'm not arguing for an algorithm change: (1) because I don't know your algorithm, and (2) because I like my high-score :) . But stars would force me out of mathematical comparisons, and just allow me to associate a positive or negative rating to the user. Anyone with higher than an 80% rating would most likely get 5 stars.
Five stars would allow me to think, wow this person contributes a lot and is well respected. 92.7 percent makes me think, well what did this person do wrong—which comment dropped or post dropped them from 95% to 92%, and what prey tell caused the .7%. Now, admittedly, I maybe more curious than some, but unless your goal is to use the percentage ratings to increase the investigation and speculation skills of members; I think that's going down the wrong path.
Shameless self promotion
Come on guys, I got a 100%, don't mark down. Mark up! Please. Pretty please.
Alliteration & Political Puns
Yes, the alliterated political puns were purposeful. A girls got to have some fun.
P.S.
Speaking of repetition, I vote for stars.
I appreciate being included in such company as those others are all great writers, but I don't think it's deserved. I've only written one post on Newsvine and I can't say it was that great to be honest; at least in comparison. However, I think you do make some good points. It does seem like the only thing to gain from sticking one's neck out is something less than a perfect rating.
Yes, this was actually my only real problem with the percentage system. To me, whether you're 90% or 100%, that's pretty much the same thing. You really just want to display the following sorts of classifications:
1. Upstanding member... healthy amount of contributions... seldom reported.
2. Upstanding member... but too new or silent to really have a reputation.
3. Active member... but controversial.
4. Active member... but controversial and confrontational.
5. Troll or spammer.
You can put almost everyone into one of these groups, but it's just really tough to represent in a clean fashion.
As others have already pointed out, knowing which category people fall under (what type of contributors they are) is more important than knowing an actual percentage score (80% vs 90% vs 92.2% vs 92.8%???), as far as filtering content, and since I think your 5 categories are appropriate Mike D., I will go ahead and vote for a 4 leaf (star) system:
0 : spammer/troll (and possibly confrontational)
1* : Active and confrontational (*)
2** : Active and controversial(*)
3*** : Upstanding but new/silent type(**)
4**** : Upstanding and good contributor (low flaggage).
(*) Differentiating between "controversial" and "confrontational" is going to be the toughest, in my opinion. Maybe another flagging category would help? Or should confrontational contributors be thrown in with the trolls and reduce this to a 3-leaf system?
(**) For this to work the Ranking system would have to consider how long it has been since a person has last posted... or maybe their leaf number should not decrease but rather change color like when an IM contact goes "away"? This could help differentiate between "good/bad" and "active/non-active" contributors. The leaf count could show the content rating, and the color could show their level of active contribution.
Just a few ideas to help you guys fine-tune this puppy. Keep up the awesome work!
Mike D. said:
1. Upstanding member... healthy amount of contributions... seldom reported.
2. Upstanding member... but too new or silent to really have a reputation.
3. Active member... but controversial.
4. Active member... but controversial and confrontational.
5. Troll or spammer.
I disagree that controversial writers should be ranked lower than writers who everyone agrees with. I think a good controversial writer can create much more interesting articles than any mainstream writer who simply reinforces beliefs that people already hold. Isn't that the point of a site like Newsvine? To encourage healthy debate between users? The Vine would become a bit dull if all of its readers agreed on everything.
I propose a slightly modified hierarchy:
1. Upstanding member... writes/seeds controversial articles which provoke an intelligent debate/discussion.
2. Upstanding member... healthy amount of contributions... seldom reported.
3. Active member... but too new or silent to really have a reputation.
4. Active member... but overly confrontational and occasionally offensive/abusive.
5. Troll or spammer.
Just to be clear, by "controversial" I don't mean "one who writes things which spark debate." I mean "one who writes things which on their face, really aren't true, but instead are designed to spark controversy through hyperbole and other means".
In the end, this sort of person is very hard to measure, but it can be done over time. This is a much more complex problem to solve than it may appear. This is the root of the difficulty:
1. You want to clearly reward people for being veterans of the site who have a great track record. These people can and should receive designations by their names (whether 5 stars or 90% approval or whatever).
2. At the same time, you need to apply negative credentials to those who abuse the system and other members of the community WITHOUT putting them in the same group as "new users".
An interesting problem indeed. Glad we're on the same page, though; I originally misunderstood what you meant.
Well, hopefully the "greenhouse" is doing its job by filtering out the abusive people before they get a chance to be published on the Vine. It seems like an excellent idea, but I must say that I don't visit the greenhouse as much as I should. I prefer to stay on the Vine and read news I know won't be questionable, although I may disagree with it. However, I can't help but feel bad for new users; if too many people are like me and rarely visit the greenhouse, it could prove difficult for new users to ever emerge from it.
Phew! I have a new respect for what it takes to be a Newsvine employee-- so many problems to be dealt with...
-Josh
We check the Greenhouse pretty frequently over here so people get swooped out rather quickly.
The question of who to rate higher, a controversial or corroborated writer, would derail this idea. I agree with you that we learn more from disagreement. I believe in constructive conflict, in which people think and bring out their best. I, however, don't think that being controversial necessarily means you would do enough to be reported.
I personally think that only extreme misinformation, sensationalism, deliberate manipulation and trolling should warrant an abuse report. You can be controversial without doing any of these.
So I would rate both a controversial and a corroborated writer the same.
I concur. What I was trying to get at is that it seems inappropriate to rate a corroborated writer as a #1 while rating a controversial writer as a #3.
-Josh
I'd imagine that rating things like the controversial writer will delve into deeper data than just moderations. Consider this: Newsvine should (assuming their database is well normalized) be able to determine:
Moreover, these datum may be further influenced by WHO is doing the moderating. A user with a high feedback rating may have a more substantive impact on the moderation process than someone with no rating or a low rating. In short, an upwards moderation by a user in the high 90s might mean more for your positive feedback rating than the same moderation by a user in the low 30s.
This formula can get very complex very fast - which is why, if Newsvine is clever, they'll run it as a nightly process rather than something that happens live - though I'd guess that's already the case.
Could we also have a "hold" feature, where if you had to go on vacation, or somewhere else, you could "hold" your account. It would go inactive, and you would not loose any points. That said, your articles would behave normally, but you couldnt post any new ones.
I like the fact you guys are considering different things. I really don't have much to contribute because most of it has already been said; I would just say give these things a go now, and if people seem unhappy down the line, then maybe adjust things a little. Good job, guys.
To anyone at Newsvine-
I like the nested commenting very much, but the more I use it (especially on articles with 100+ comments) the more I feel the need for a "Jump to First New Comment" button up at the top somewhere. On huge articles when my conversation tracker tells me there is one new comment, I have to sit and page through what could be dozens of pages looking for the "New" tag on one comment.
Thanks,
-Josh
Hi, Josh.
When you click to the article from the tracker, it should automatically take you to the first new comment. At the end of the URL in the address bar, you should see something like '#last_x' which will take you there. If that's not the case, hit us up with an email that includes the URL that's in your address bar after clicking on the tracker.
- also Josh
This little subthread brings up an interesting trend I have seen. I keep emailing comments and suggestions to newsvine and often get a reply of "we already do this" or "if you follow these steps, it already does what you want".
Well, this says to me that there is an issue.
The issue is not thinking of or responding to suggestions/ideas/issues ... it is that the "fixes" or "features" are not as intuitive as the end users would like.
You make stuff like the tracker button in a place where it is not very noticeable (especially in a long post), or the "new" comment market.
Just a thought ... the overall design is very beautiful, but might could use some work on the functionality & intuitiveness.
Check your "suggestions/bugs" emails and see how many have had a reply similar to what I discribed above.
Hi Nation. Actually, what J. Halbert is suggesting really *is* already in place and should already be doing that automatically, as Josh mentioned. If it's not, then something is wrong. When you click an article in your conversation tracker, you should automatically jump to the first new comment. No figuring out necessary on the part of the user.
Mike - I think what he's suggesting is that it shouldn't necessarily be something we have to use that little javascript widget to get to. If I go back to an article that I've read before, why not flag the comments that are since my last read for me?
On a related note - I get some errors on that javascript widget's count of new comments. It seems reluctant to clear out old comments once I've read them.
Hi Killfile. That's actually not possible, unfortunately, because the "new" icons and the jump functionality are created by using a named anchor. So if you were to hit the page just by typing in a normal URL, you'd just go to the top of the page. Since the Conversation Tracker contains information which it appends to the end of the URL, that's what allows this functionality to occur.
Mike D.:
That is perfectly understandable. Is there any chance of moving back to the flat comment structure or is the single child layer here for good? I have mixed feelings on the new comments, it is nice that conversations can now be separated, or at least in more cases that before. But at least in my experience getting the feel of the whole thread has been harder. I guess I am not sure which is best, but I am wondering if there is a chance of moving back to the old ways, or are they long gone?
People settle into one system, then when something changes, it creates lots of un-ease. You would find if it were changed back, you would want this system.
Paradox460: That could very well be true.
Actually, it seems nobody understood the point I was making.
It has less to do with the comment system and more to do with user expectations.
The comment and reply that J. Halbert & josh had reminds me of several times I have sent in suggestions and the reply I receive from a Newsvine staffer.
I will say something like "I wish you had " or "Please add " and the reply email will be:
"there is this function, please use the 'track' button just below the comment field."
or
"When clicking a watchlist tag there is a green indicator arrow denoting an item that is new."
These are items that people want ... but for some reason I am not picking up on them ... and it seems that others are not either.
This is not a "dumb user" issue as much as it is a "features & functionality not being where the user would expect". Go to the top of this page. Try to get the article tracked if you have not already commented in it .... do you see anything that would cause you to think "if I click this then the comments will start being tracked?
The design seems to be mostly "what looks purity" and not "what is useful". You are designing a new vending machine ... you need to take more into account what your user expects to see and happen when they touch/click different parts.
And to quote a song:
I'm not trying to be a nuisance
I just think we can do better than this
That was simply my two cents
You can, you can take it or leave it.
You are doing many good thigns, but good is the enemy of great.
I would imagine that these guys are working hard enough on major features and ambitious projects at this point that your emails complaining about minor details is understandably not high on their priority lists.
Give them a break, they have just a small team working on this incredibly innovative site. I'm sure they physically can't satisfy every single need of everyone as immediately as you would like. The rapid progress they have been making recently deserves great respect.
ok, I am going to try one more time, then give up on this thread.
There are so many features ... that individuals think "I want to do ". They go and look to do that and cannot find the way to do it easily.
Now the thing is ... often the is already built into Newsvine, but the location or process of action is not intuitive enough for the individual to find it.
This is not an issue that is just going to go away. Someone has to put cautious effort into asking questions like
"what does the user expect?"
"where would the user look for ?"
"What would the user want to do when they read this?"
"How many people have asked us about ? Maybe the button> is in the wrong place."
I am not sitting here saying "Newsvine is dumb". I really do see potential in it ... I have also attempted to be very careful in the words I choose, because I understand about 70% of the time people read into online posts tone and such that is not what the user attempted to say. This is just something I have noticed over time ... and every once and a while I see someone else post something similar to what I said above (the "I wish I could ." and receive a "You can, it's process>.")
Graphically Newsvine is great, but on the intuitive meter ... it could use some work. I ask again. Go to the top of the page. Assume you have not commented in this thread and want to track it .... you are a newer user ... how do you track this thread without commenting in it? (I know the answer, but put yourself in that mindset and go and look for the button at the top).
Now go back to the previous post I made and read the last blockquote and sentence.
I just realized that if you do a "<" then some text then ">" newsvine cuts out the whole thing ... so my above message might not make total since. Oh well.
Nation: I do understand what you are saying, but perhaps my response wasn't very useful. The answer to your concern is "yes, we need to always strive to make things as useable as possible"... and we do. There are a lot of things around here which have already gone through the iterations you are suggesting, and probably a few things (like some of the more advanced features) which could be explained and presented better. It's a bit of a paradox though as you really need to be careful about what you show to new users. Show too many controls and users won't even feel comfortable doing simple things like reading stories ("why is this 'Track' button sitting here at the top of this story? I just wanna read."). Show too few controls and users will never discover the advanced functionality of the site. Our philosophy has been to try and put every control in context. For instance, why would you want to track a conversation? Because you're interested in the comments. If you're interested in the comments, you would theoretically have read them. And so the Track button is at the bottom of the comments... right where you would have left off after you were done reading.
Keep the suggestions coming though... everyone uses sites in different ways, and when you're doing things no other sites are doing (like the ability to track conversations), you are bound to get varying reactions.
By the way, not to split hairs, but it's an important distinction: The Voltaire quote is actually "perfection is the enemy of good"... which makes a lot more sense than the other way around. :)
I was pulling more from the book Good to Great by Jim Collins (you might consider looking into it; one of the best business books out there) and other business/strategy/org theory stuff I have studied.
"perfection is the enemy of good" and "good is the enemy of great" are two drastically different statements, perfection != great and they address two different situations that people/organizations fall into.
Aha... looks like an interesting book. Serves me right for trying to correct a quotation. That Voltaire quote happens to be one of my favorites so I mistakenly thought you were referring to it.
Anyway, keep the suggestions coming. We roll code multiple times per week over here and the only way we can know what needs work is when users like you point it out.
One way you could adjust these features is having a "Simple Newsvine" option, like Simple Finder in Mac OS. Users sign up, and there is a check box that is to "Enable Simple Newsvine", with an explanation. Then, there could also be a drop down pane, where users can disable and enable features at will, under the prefs. By default, Simple Newsvine would be on, but it wouldn't affect already existing users, and users could turn it off at any time.
Here is an example of a link that did not jump to the new comment when I selected it from the conversation tracker:
It still has this "#last_1" tag, but it did not work. Most of the time, it does, but on occasion, it won't.
And to all of you who work for Newsvine, please don't misunderstand what I am saying. I am hardly trying to say that Newsvine is in any way "bad" or that I am not satisfied with it. On the contrary, this is without a doubt the best and most intuitive news site I have ever used. Anything I say about Newsvine is meant to be constructive, whether it be a compliment or criticism.
Thanks and keep up the great work!
-Josh
That's interesting... when the link showed up above, the "#last_1" is a not part of the link.
Does this mean anything to anyone?
-Josh
In a URL %20 is encoded to represent a space -- if everything is being done right. I'd guess that your URL, which is based on the title of the piece in question isn't being properly encoded.
If the title were to be "does the us have one? " (note the space after the question mark) instead of "does the us have on?" and Newsvine isn't trimming the trailing spaces off before appending the "#last" it would definitely cause your problem.
Go to the top of the page. Assume you have not commented in this thread and want to track it .... you are a newer user ... how do you track this thread without commenting in it? (I know the answer, but put yourself in that mindset and go and look for the button at the top).
Nation, I feel your pain on this intuitiveness point. In fact, I didn't notice the conversation tracker at the top of the page at all until someone commented on one of my posts and the zero changed to a higher number. The sudden difference happened to catch my eye; otherwise, I may have remained clueless for a lot longer. This is not to mention the fact that I wasted quite a bit of time searching around for a way to receive email notifications about comments similar to the functionality in other blogging systems.
Can you please tell me how to track comments for a post that you have not commented on personally? I was just wondering about this very thing last night, lamenting having to leave a tepid remark on every post I'm interested in following just to get the automatic comment updates through conversation tracker. Thank you.
Chandra W.
There is a small "track" button at the bottom of the page. Under the comment box & all the comments.
So press your "end" button to go to the end of the page & look for it.
I emailed newsvine with basically "I want a way to track comments without commenting in the area" and their reply was:
"Nation, there is this function, please use the 'track' button just
below the comment field."
That's great, but it seems I am not the only one who totally missed it.
To Chandra W., Nation, and anyone else having trouble tracking conversations.
A quick 2-click search in the help section found me this page which states, "You can also add an item (to the conversation tracker) without commenting by clicking the "Track" button underneath the comment text box."
If you scroll down past all the comments and past your comment input box, you will (should) see a "track" button. Click on this button, and you can track any conversation without commenting yourself, or even voting on an article.
If you can't figure out how to do something, you should try the "help" section first. If it's not there and you get a response from the staff saying "the vine already does this", then there might be a problem where the help section is not updated as often as the Newsvine Blog, which talks about all new features in detail when they are released.
Nation and Rodo, thanks for your very helpful replies.
Help Sections are always useful, but what is occasionally even better and more user-friendly is some indication nearby a functional interface or tool that quickly explains what the functionality is and how to implement it, identical to the "What's This" question mark in parentheses that Newsvine already employs in the leftmost column to explain the purpose/function/definition of Feeds, Watchlists, Top Contributors, etc..
Making those "What's This" indicators more widespread throughout the page could help simplify life for those users who don't like weeding through a whole bunch of time-consuming FAQs, for whatever reason(s), or are simply unaware of how to get to them, again, for whatever reason(s).
Also, I have looked directly at that Track button at the bottom of comment threads so many times it's not even funny, but most often on relatively short pages. It would be nice if the button were located immediately beneath the article, as well, or somewhere near the top of the page to make it more visible. Once a thread gets to a certain length, it gets that much harder for uninformed users to immediately locate the button.
I agree Chandra W. I pretty much hate going through faqs and was surprised to see how short and sweet the help section was (which is why I was able to find the answer in only 2 clicks). Like the rest of the site, they've done a good job of keeping the help section clean and tidy. Hopefully this won't become an issue as more and more features are implemented.
Only the staff can truly gauge the need for improvements in this area since they are the ones that receive all the "how do you" and "where do I find" questions from users. It's been awhile since I signed up to the site, so they might have this already, but if a lot of new users are asking similar questions, maybe an "introduction page" similar to gmail's (when you create an account and login for the first time) could help new users understand the features and little details of the site?
Thanks for a great site guys, keep up the awesome work!
I'm not sure about the abuse reports being a good thing to factor in. Say I'm a rabid big-ender and some narrow-minded small-ender logs in and reads my stuff. Shocked at my big-ended diatribes, he and his other small-ender buddies join the site and slam my articles with repeated requests to boot me entirely from the site. Maybe they have a corporate backer to apply additional, moderate pressure. Or lawyers.
Meanwhile, my feedback number goes to crap. How about instead count up the number of approvals people have given me, only? All the big-enders showing up as an approval posse would be one reason there...
How about granting people the ability to approve based on their contributions? Well, that creates a bit of an elite core group... aaaagh, there are no easy solutions, so how about letting us know what got flagged as inappropriate and have those marks reviewed by staff to see if they should be removed or changed into "controversy!" ratings.
There are people who are abusive. Report them and end their abuses. There are people who are controversial. Reward them with controversy points, so we can get an idea about who we might "love to hate" on the site.
You're in Easy Mode. If you prefer, you can use XHTML Mode instead. |