32 people says things!

1. I find the whole rating system on Stylegala a little bit like sitting in a focus group with one very loud annoying bloke steering the group.

   What I mean is they are no indication of success of a design, they are subjective and a pretty blunt tool, with no criteria, to rank sites. The comments are much more interesting and give more context to peoples judgements.

   It is interesting to note that most of the sites in the top ten on Stylegala are pretty well known bloggers. Success breeds success and all that.

   By Mark Boulton on May 13, 2005 9:28 am

2. Honestly, as I was writing my post yesterday, I was thinking that Style Gala, or any rating system, should be based on more objective rather than subjective things.

   How is the typography? Does the layout make sense? Does the design support the goals of the site or is it design for design’s sake? (I know, sometimes, the design is the only goal of the site.)

   Then there is the current way people judge sites. “I don’t like that color.”, “I hate arial!”, “It’s too simple.” or whatever they might say.

   In my opinion, if you’re reviewing a site, or providing your input and use the word “I”, you’re probably not being objective.

   In conclusion, Paul, I think the reason that there are no 10’s is that everybody is voting what they personally like or don’t like. They aren’t voting whether the design is a successful design or not. As such, even the most amazing sites get dragged down.

   By Garrett on May 13, 2005 10:09 am

3. I agree with both of you and it’s a shame that it works out like this, but in all honesty any voting or rating system is going to run into these problems, unless there are clear guidelines stated.

   Another negative about the system is that new designers might see one great site with a 4.6 rating and think that it isn’t the proper way to design. This is a shame because many times you end up with designers trying to overdesign a site simply because they feel that is the way to go.

   I am not saying that the rating system should be disposed of, but David and the Stylegala crew could do well in having a set of basic guidelines to follow (not rules).

   But the comments are usually where it’s at.

   By Scrivs on May 13, 2005 10:14 am

4. Weekly Standards used to have a nice rating system with several variables: design, usability, etc. Let’s hope Mr. Archer returns that feature in the relaunch.

   By huphtur on May 13, 2005 10:44 am

5. Generally speaking I don’t participate in completely open-juried contests or sites like Stylegala because every shmo’s vote counts the same. Why should a skilled and well-established designer’s opinion count as much as some nutjob who thinks sites should have animated backgrounds to be “eye-catching?”

   Also, people don’t vote on the actual criteria. I wouldn’t be surprised if a number of people took points off their rating of, or even gave a zero to, the site above because of disagreement with the content of the site or even the use of an obscenity in the text.

   This is not limited to design sites. Other community-moderated sites have the same problem. Here’s my discussion about Kuro5hin.

   By Thud on May 13, 2005 11:01 am

6. Do I find myself needing to defend myself for no real reason? I am not sure but perhaps you could tell me what “Oxton’s bad bay” means because I haven’t got a clue.

   By John Oxton on May 13, 2005 11:09 am

7. Looking back at one of my comments, I realize I actually made a mistake as I was one who said Garrett’s site was too simple to necessarily make it into Style Gala’s archives.

   Originally, I thought all the vault like galleries out there were more for showing fantastic and unbelievable designs with CSS, hence why ALOT of designs get rejected…but then I thought about it more, even after my comments on Garretts site, that there is so much more to design then taking it to the edge.

   I think for a time being, the vault like sites were only accepting outrageous designs, but now it seems they are cooling down a little and accepting sites that make sense. Garrett’s site ‘makes sense’. Its simple, but looking back its very simple and usable. Does that make it a good design. Yes it does. I “do” think its good to show off simple sites like these, whether its complex css or simple css.

   My original comments were ignorant to the fact that I was thinking stylegala, cssvault, etc.. were more for complex css designs. That is wrong and I do apologize.

   By Bryan on May 13, 2005 11:11 am

8. Whoops that should say “bad boy” as in badass site.

   It means your site rocks. Simple as that. If you need to defend your great sense of design then by all means have at it :-)

   Thud: That’s a spot-on analysis of voting systems.

   By Scrivs on May 13, 2005 11:13 am

9. Thanks huphtur! I was scrolling down to throw in my two cents on the way I chose to use voting with TWS and I see you’ve already done it for me ;) I liked my solution, but sometimes I worried that it could have been a little bit of overkill. Like making people vote in all 5 “design categories” everytime.

   I do think a lot more work should go into our present voting and comment systems — especially for site reviews. Maybe voting in 5 out of 10 areas and providing a little comment about why you voted would be a good approach.

   It’s an interesting discussion. What would be ya’ll’s perfect voting system? Something fair that would produce interesting numbers and insights. How would you do it?

   By B. Adam on May 13, 2005 11:13 am

10. Hold on Scrivs! I’ve ordered some new headers from Amazon but they haven’t arrived yet. ;)

    By Malarkey on May 13, 2005 11:15 am

11. After thinking about commenting, I’m starting to feel there needs to be a slight barrier to entry. Enabling people to just click a box to vote doesn’t make people justify their decisions.

    A system w/ different categories, and ratings for each category would be great. Then go a step further and make people justify their decisions w/ writing. Naturally, this would bring in fewer votes, but they would hopefully be more objective. It’s a tradoff either way, and I don’t think there is a perfect system short of having a set panel of judges that are basing their decisions on consistent and objective criteria. But that’s not democratic.

    By Garrett on May 13, 2005 11:21 am

12. Andy: Too cheap to get overnight shipping? :-P

    Garrett: Having less comments and votes isn’t a bad thing at all if that lessens the signal vs. noise.

    Cliche-time:

    Less is more.

    But thinking about your system I definitely could see a process where you vote and have to justify your vote with some commentary. Your vote and comments are submitted, but not immediately published until they have been moderated properly. Adds to the overhead, but makes for higher quality I believe. Damn, I have 1,000 more ways to improve things…time for another site.

    By Scrivs on May 13, 2005 11:27 am

13. Opinions about aesthetics vary from person to person. Personally I’m not overly keen on Garretts site.

    I think the bright, oversized headings detract from the content. If the content font size was increased it would be a lot more balanced.

    My current favorite site was mentioned here the other day (http://www.penandthink.com/niggle/). This site uses bold colour, along with massive fonts, yet it’s a lot more readable.

    I’m a fan of Andy’s site; although I agree that the Impact headings aren’t nice to look at. I use Windows XP but have antialised text turned on, for some reason the headers in Andy’s design remain aliased. Not pretty.

    I think the idea of restricting voting based on design skill is rather elitist. You don’t need to be good at design to know wether something looks and works well. I can’t make cars but I know what I like to drive and be seen in.

    There’s no reason not to change the voting system though - adding different catergories would be a good start (as mentioned).

    By ben on May 13, 2005 11:33 am

14. The site is a 5.0 for a few reasons in my book:

    1. There is no universal appeal to it. This is not necessarily a bad thing, but it’s true. Usually the things in life that people REALLY REALLY love are so original and new that others are bound to hate it and thus drag any “overall score” down if the detractors are given votes. Some people seem to really like Garrett’s design and others think it’s underdesigned, derivative, and blah. A 5.0 is a perfect outcome between those two groups.

    2. Not to diss the site or anything, but one thing that strikes me about it as a designer is that it seems like one of the most easily reproducible sites I’ve ever seen. Give me about 20 minutes and I can clone it without using any of Garrett’s code or graphics. There’s something about this quality that subconsciously or consciously affects people’s opinions of the site. “If I can reproduce it in 20 minutes, why should it be considered great?” It’s the same feeling you get when you walk into an art museum and see a 30 foot by 30 foot canvas painted in just a single coat of blue. Hmmmm, why is this art again? I could do that!

    3. Despite what some people say, I don’t find the site easy to navigate at all. It’s a bit underdesigned in that regard. I want to know more about Garrett so I look for an “About Garrett” link somewhere. There is none. Only a little while later do I realize that I must click on his name at the top of the page to accomplish this. Cute, yes. Minimal, yes. But not intuitive. I applaud all of the experimentation and rule-breaking that people do on the web, but navigation is sort of where I draw the line. Experiment with your content and your individual pages, but don’t try to break the navigational habits of your users. People expect a top nav or a side nav so you’re inhibiting their ability to navigate your site if you take that out. Now, with a small site like a personal blog, it’s not a big deal… but that’s just the feeling I get when trying to click around on this site.

    By Mike D. on May 13, 2005 11:36 am

15. Ahh right thanks for the clarification! :)

    StyleGala’s voting system is very subjective for sure and whilst I am chuffed to bits to find myself with a 7, I think that was simply the flowers…or dare I say it, the WOW! factor. Had I stuck to my simple version (available on the site) then I probably wouldn’t even be on there.

    It’s also difficult because some sites are undertaken by a team of designers, others by individuals and I don’t think it’s a fair game in that sense (except when a single designer whoops the ass of a big team of course!).

    I would like to see a site dedicated to web designers work overall, i.e. not one site but many sites, or in other words, showcase consitently brilliant designers (actually I’ll just copyright that thought right now!!).

    That was a bit of waffle but you get the idea.

    what sites rank highest on my list?

    SmallTransport.com
    kingcosmonaut.de

    and http://web.burza.hr deserves to be where it is in my opinion.

    By John Oxton on May 13, 2005 11:44 am

16. Why not just make the voting system more diverse? So instead of just saying the whole site is a 6 or a 3 give the voters the criteria. Give them options.

    If you want the voting to be based on the elements of design let them vote on colors, layout, and typography separately.

    That way an algorithm can be run on the votes for those criteria and a total score can be created. Wouldn’t that be a bit easier than making people adhere to specific guidelines?

    By Mike on May 13, 2005 11:45 am

17. Apparently I’m a little late to the game, again. Damn clients always bothering me instead of letting me post comments on blogs.

    By Mike on May 13, 2005 11:51 am

18. I think that was simply the flowers…or dare I say it, the WOW! factor. Had I stuck to my simple version (available on the site) then I probably wouldn’t even be on there.

    Good point. I love the flowers (who doesn’t?) but strip them, leave solid color in place, and what’s it going to score?

    That said, the vault-esque galleries are good places to collect wow-factor design samples. If wow is what it takes to convince developers world over to adhere to web standards, I’m all for it!

    But if valid markup and far-reaching CSS practices are the norm, at least in the sphere of concern for whoever’s doing the submissions, what do those gallery sites become? Roughly the same thing as linkdup et. al. Not to say that’s bad.

    By Joseph Wain on May 13, 2005 12:03 pm

19. Good thoughts people. Now, just to clear my throat regarding the “why does all those kick-ass sites average 4.5?”. It’s all about basic math and statistics. If the numbers would be picked randomly there would only take a certain amount of numbers before the average reaches 5.0

    In this case, the numbers are not random, but peoples opinion are so diverse they are sometimes darn close to random. That is why a site can start off with 8.0 after 1 week and then drop to 6.0 after 3 months. The more votes, the closer to 5 it gets. Basic math. A short-term solution would probably be to have more decimals in the rating.

    The rating system is meant to be a “public voice” (I love that term). Any designer, blogger, medical student or ski instructor have the same impact on the site’s rating. Fair and square. The downside of this is of course lack of well-thought and motivated ratings designer to designer. That’s where the comments come through.

    Categorized votes seems like an awesome idea. Who do I have to pay to implement that?

    By Smilegala on May 13, 2005 12:04 pm

20. Some people might be looking for eye-candy graphics, others for CSS techniques pushed to the max, others for salesmanship, others for usability. That’s why the ratings might look unfair on some cases. Design and beauty will always be in the eye of the beholder.

    I’ll take functionality over eye-candy anytime, which is why I posted Garret’s website in the first place - as a reminder to everyone that design isn’t all about graphics.

    Remember what Leonardo da Vinci said: “Simplicity is the ultimate sophistication”.

    By Lucian S. on May 13, 2005 12:46 pm

21. Aww, shit man, it’s all about personal preference. Aesthetics and bullshit. Nothing new here. I like quite a few sites over there, but I’m not to fond of web.burza - Superhero Edition which is currently holding the top spot. Looks nice, but I’ll take Garrett’s site over that any day.

    You could wonder why someone would want a higher rating. I mean, that could just mean you’re pleasing to the “norm” and we all know how un-cool that is.

    Then again, what do I know, I can’t even get into Stylegala. ;)

    By Keith on May 13, 2005 2:27 pm

22. I haven’t followed the comments from the beginning of this post, and don’t have time to read them, so sorry if this is repetetive… but I really think gallery sites should have a “survey” based rating system where they don’t simply give a blanket rank to site, but are forced to rank specific elements, like design, creativity, typography, color choices, usability, etc.
    This way, a more objective rating will result.
    I think most of what people are voting on at Stylegala is the design itself, and that is very much directed to personal taste. If the other elements of “design” were included in the rating, we’d probably see some 9’s at least.

    By Jay Jones on May 13, 2005 3:50 pm

23. I agree with Jay (#22). Ranking sites on different criteria, as well as providing an overall score would be useful.

    It would also help people who are coming to the site looking for inspiration; want to check out some great typography? Go here. Want to see which sites are the most usable? Then check out this category. And so on.

    This is done pretty well in the video game review sphere, where games are reviewed for gameplay, graphics, sound, etc. Check out http://www.gamespot.com/pc/rpg/guildwars/review.html and http://pc.ign.com/articles/612/612079p3.html for examples.

    By Christian Watson on May 13, 2005 7:33 pm

24. After a day of experimentation with both sIFR and image replacement, the site now has shiny, new, graphical Impact headers which I hope you will find less a strain on the eyes. After all, it’s legibility which matters on a content based site and I hope that I’ve struck a reasonable compromise.

    The IE Ska design uses plain text Impact, but at a size which appears to work on Windows PCs without ClearType running.

    I want to say a huge thank-you for all the kind comments about my new design. They have been (almost) universally positive and that puts a really big smile on my face.

    (@ Scrivs: I love Amazon, my new headers came a day early even though I’d selected ‘Deliver by Carrier Pidgeon’ ;) )

    By Malarkey on May 14, 2005 4:30 am

25. Ah damn Andy, that makes it 100x easier to read now. Greatly appreciated.

    By Scrivs on May 14, 2005 8:30 am

26. @Garrett (#11):

    Dis-Vision is a close example of what you’ve described here Garret. Although inactive, Dis-Vision has the right ideas executed quite nicely. The founders just need a little more passion in the project for it to flourish.

    By Adam Bouskila on May 14, 2005 1:49 pm

27. The problem is that equal portions fanboys and jealous jerks are rating the sites. They all rate the sites 10 or 1, respectively. The small slice of honest reviewers in the middle make up the deviation from 5.5, slightly raising or lowering the average depending on what the more popular opinion is.

    That’s my theory, anyway. :)

    By Scott on May 14, 2005 5:44 pm

28. I think an average under 8 is the direct result of tastes being too subjective with people. I love minimal designs like you, it pleases me more than complicated designs with a gazillion graphics tagged on. Then there are the relative experience of the people judging, from the inexperienced to the overtly experienced, and that mixes it up even more.

    The most important lesson I’ve learn in web design is that you can’t please everyone, and not even most people, what counts is that you try to, at least.

    By Matt Burris on May 15, 2005 12:00 am

29. To be honest I’ve never understood the point of allowing anyone to rate others designs at all.

    Whats it for? What does it prove?

    I like all types of design, both heavy and minimalist. Why have a preference when good typography, use of imagery, use of space etc is equally applicable to both schools of design. Even more significantly, why ‘rate’ it in a way thats designed solely to pigeonhole it. And further, why worry about what any design recieves as a rating? Its an entirely meaningless statistic. There will always be people who like a design and people who don’t. If the majority don’t like a design does that make it bad? No. When did we stop designing for markets and start designing for the approval of other designers?

    Don’t get me wrong, the times I got mentions on CSSBeauty and CSSVault really made my day(s) (and its been quite awhile since I got a listing!) and its nice to be well thought of but the idea of rating a design based on a set of aesthetics that are entirely personal seems, well, a little daft.

    Some of the comments that are made betray the experience level of the commenter in question. Following Garrett’s listing some really bizarre comments were left about not using cutting edge techniques which seemed to me to be the antihesis of what ‘good’ design is about. When did it become about cutting edge technique and stop being about user experience?

    By Kev on May 16, 2005 4:26 am

30. Seems like we’re all saying the same things over and over. I wonder how that applies to the subject…

    Individual voices are priceless; however, when our voices work together and build off of one-another, we achieve real results.

    The answer, in my mind, is to better enhance the interactive of such “voting” sites, so as to inter-twine the individual voices together. As was previously stated, “the comments are where it’s at”.

    Would anyone like to take a stab at a site model that would merge voting, comments, author feedback, etc., all into one usable structure that everyone could just love and want to use? Like a Mac?

    Haha, okay… that was a very subjective 3-word sentence. I digress.

    A new model is definitely what we need to revolutionize this space. Adding criteria and rules may be a decent short-term solution, but people need to fall in love with what they engage on the web in order to be a significant part of the process. Let’s reinvent this totally.

    I wish I could think of some decent examples, but I have yet to sleep ant it’s 6am. Don’t ask.

    Anyone want to begin formulating a real-life skeleton for my early morning foundation/philosophy/challenge?

    By Bradley on May 16, 2005 6:05 am

31. One thought is normalization of scores (a la HoN) or normalization per voter even.

    But why not just reduce the ratings to a [this is good] or [this is bad] option and add a tagging system for categorization?

    Or, better yet, get rid of “ratings” go totally w/ tagging and a bookmarking/fabs system, so that if you want something you want to reference later, you’ll put it there (making it personally useful - a key for effective folksonomy development).

    Different listings can then be easily dynamically created (or even further analysis generated from timestamps, etc.).

    By Leonard Lin on May 16, 2005 2:11 pm

32. Might be time to get off WoW and update that spam filter, Paul… :-)

    By JC on February 1, 2006 9:34 am

33. Subscribe to comments via RSS!