Design 101 tells you that you need to test your work on people as soon as possible - a paper prototype or just anything that has relevance to the planned product.
A week after the ALPHA release we introduced the Second Brain idea and ALPHA solution to the various departments in Making Waves. We presented the functionality and gave everybody a chance to sign up and do a test drive.
Then the feedback started coming in - bug reports, questions about the concepts, ideas for improvements. Most people were positive. There are lots of people who really like the idea of Second Brain, and who want us to succeed. These people are usually more encouraging than critical of what we are doing. And that's fine, but encouragement and positivity sometimes inhibits people from focusing on how to improve something.
The best feedback we got from the people who were able to challenge
the concept and our proposed ALPHA solution.This type of feedback,
although it may hurt a little bit when you receive it, is almost
impossible to overestimate. A few challenging questions from the interaction designer, a few "I don't get it" comments from a technologist, or a "I don't know if I need this" from the management, makes you question the fundamentals of what you are doing - it fuels your creative juices.
To make a long story short - the positive feedback is nice and encouraging, but the negative, or rather, constructive feedback, is ten times more powerful and necessary. While it is more comfortable to talk to people who dig what you do, as a designer of anything new, you should always seek people who can challenge your ideas and your solution - the earlier the better. Be brave.
In retrospect, I think we could have asked for feedback earlier in the ALPHA stage - for example from a paper-prototype - and maybe save some development time and entered the BETA stage earlier.