Recent Changes - Search:
  • Welcome 23.20.162.200

'We have 5 guest/s online'

Content Areas

Other Areas

Our Sites

Save to del.icio.us Save to LinkAGoGo Save to Ma.gnolia Save to Yahoo! My Web

edit SideBar

Evaluation And Designing

Independent, objective and consumer focused product performance testing, benchmarking and consumer user trials are beneficial in product development, product sourcing and product placing decisions. They will let you know if the product is fit for its purpose, easy to use and fulfils a 'consumer need', which will help to increase sales, improve brand awareness and increase customer loyalty.

Products that fall short of customer expectation lead to undesired after sales costs such as warranty claims, product returns, unsold, unwanted products, and, possibly product recalls and corrective actions. In short, performance testing, benchmarking and user trials will help avoid all that.

Performance Testing evaluates the product from the user’s point of view. It answers the following basic questions: ‘Does the product function as it should’? and ‘Are all the features beneficial to the user’?

Through benchmarking, you will find out how your product compares with its competitors. It helps you to quantify where your product excels.

( Ref: http://www.uk.intertek-etlsemko.com/en/ppf/page_id/35/level2_id/75/level1_id/55/level3.asp )

7.1.1 Outline the general criteria used to evaluate products.

Consider performance, reliability, ease of use, safety, aesthetics, materials, construction and cost.

Performance:

Does the product meet the consumers needs?.

Reliability:

Does the product have a useful life. Does it work when it needs to? Does it need many repairs?

Ease of use:

Is the product easy to use? Is the manual easy to understand? i.e. user-friendliness. A product that is too difficult to use would result in consumer frustration.

Safety:

Is the product safe for the user? Is there a chance of injury or a potential of injury. If a product is hard and repetitive to use then this cause the user to tire and loose concentration, eventually injury could be caused.

Aesthetics:

Does the user get pleasure from the product? How does the product look, feel, sound,taste or smell.

Materials:

Are the materials durable? What properties are important? Are they available locally? Are they 'green'?

Construction:

Is the product well construction and not likely to fall apart. For manufacturer is it easy and cheap to produce which will affect the price of the product or profit.

Cost:

Value for money or cost effectiveness

7.1.2 Explain how the criteria used to evaluate products will vary depending on the purpose of the evaluation.

For example, crash-testing cars is done in relation to safety only.

Every evaluation has a different purpose, the different purpose means different criteria are tested. A stress test will not be used to evaluate aesthetics but it will be used to evaluate tensile strength. Different products having different criteria, require different tests. Crash tests are only effective for transportation vehicles.

Crash test for Safety

7.1.3 Apply the general criteria to evaluate products.

Application of the general criteria to evaluate mobile phones based on a scale of 1 to ten, ten being the highest or best:

ProductPerformanceReliabilityEase of useSafetyAestheticsMaterialsConstructionCost
iphone7.57.58NA7.9888
iphone 3G998NA999.510
Blackberry Curve 83007.58.55NA5885
Blackberry Bold 90008.598NA898.56
Activity
Carry out an evaluation using the criteria explained above of your and another students mobile phone. First, check out some online review magazines (i.e. Choice or Consumer report, etc for consumer electronics ... to get a feel of how an evaluation/review is reported.

7.1.4 Explain the use of qualitative and/or quantitative tests, models and experiments used to evaluate ideas at the design development stage (developing chosen solution) of the design cycle.

Consider the use of scale models to evaluate shape, form and proportion; materials tests; construction technique tests, and so on.

7.1.5 Define literature search.

The use of consumer reports and newspaper items to follow historical development. Useful sources of information could include CD-ROMs, such as encyclopaedias and newspapers, or more specific disks, subject-specific magazines and manufacturers’ information.

A systematic and sometimes exhaustive search for published material on a specific topic. for example from books, catalogues, encyclopaedias, dictionaries, atlases, bibliographies, handbooks, manuals and academic websites etc.

7.1.6 Describe one advantage and one disadvantage of literature search for data collection.

Many sources of information are available, but there may be an abundance of data, which can be too time-consuming.

Literature search, with the increase in technology and communication has become very useful. It is has enabled us to get information in no time. There are so many sources , with a lot of information, that the user will have to search throughout all the data given and finally spotting out the aspects needed. Literature search is secondary data (receiving data from an outside source already collected). The user must evaluate the sources it check if the data is current, has any errors or inaccuracies - whether the data is dependable and can be verified. Presence of bias in the data.

Advantages:

  • It’s cheaper and faster than doing your own research
  • Data is more flexible (eg. you might only use specific figures from a table)
  • Easy access to all kinds of data due to technological advancement (readily available)

Disadvantages:

  • Finding relevant data might be a challenge
  • Might not always be accurate
  • Could be biased.
  • Data of large samples can sometimes involve large data, making it difficult and complex to understand.

7.1.7 Evaluate the importance of ICT in aiding literature searching.

Consider access to information, speed, costs, storage and security.

The use of ICT today in aiding literature searching cannot be missed. This is due to a series of practical reasons. Through ICT, the access to information is not only fast, it is also precise and straightforward. ICT can allow endless data to be stored, which adds to its practicality because several books can be stored and searched through very easily. However, there might be some problems with security which will depends on the safety of the involved person's computer and internet. Another possible disadvantage might be the lack of precision or depth of information. Because of the continuous improvement in information technology and computers, ICT ameliorates at the same pace. This has enabled it to be faster and more cost efficient as well as to become more secure and have more storage.

7.1.8 Define user trial.

The observation of people using a product and collection of comments from people who have used a product.

User trials will assist in finding out how a product performs and how users view its performance. It will help to make sure that it is a functional and appealing product. User trials can also help to understand why the product does not sell well as you had hoped.

An example of user trials is beta testing (for games). This is when copies of a pre-released game are sent out to testing individuals to evaluate performance, functionality, and quality to ensure that they are appropriate for the target market, but as well as free of errors. When a beta tester finds an error, the individual will report the error and the producer will review the error and make corrections.

7.1.9 Describe one advantage and one disadvantage of a user trial to collect data.

The “user” is a non-specialist, which makes trials easier and cost-effective. However, users may carry out tasks in different ways from those expected and be inexperienced.

Advantages

  • The user is a non-specialist, therefore, their required fees will be low
  • The user represents a typical consumer, and can give good insight into how a regular person would use or understand the product in testing.
  • The non-specialist users opinion, can represent what typical average consumers would think of the new product, whether they would 'like' it or not.

Disadvantages

  • The user is not well trained and therefore may not use the product in the same way it is supposed to be used.
  • The users opinion may be biased
  • May not accurately reflect the rest of the consumer base.

7.1.10 Define user research.

Obtaining users’ responses.

User research can be said to be consisting of two types, namely usability testing and ethnographic field research. Usability testing focuses on the characteristics of the user's relation to a product that can be measured. To determine how usable a product is, standardized tests will often be used in order to guarantee an array of quantitative data. The results of usability testing will enable an insight in trends in user behavior and will consequently show both issues with and successes of a product. Ethnographic research techniques on the contrary are based on the observation of users in a real-world setting. The observation of software users trying to meet the objectives of that software for example can take a lot of time but will also lead to valuable qualitative information about the usefulness of a product. The data acquired through ethnographic techniques helps in the development of solutions to the problem areas pointed out by the above mentioned usability testing.

7.1.11 Describe one advantage and one disadvantage of user research to collect data.

Advantage:

Because of the qualitative nature of the data, research results will allow a quick and effective discovery of problem points of a product. For example any problem using a programme will result into immediate reaction from the Designer in order to make it more user friendly/helpful
Advantage: it is primary research. Information is first hand and gained from the inside, so should be truthful and valid.

Disadvantage:

The use of such type of qualitative feedback as a way to assess the overall concept of or point out new ideas for the product can cause certain complications because it may be vague or unspecified. As an example If 60% of the interviewed people say this restaurant cooks good food. Would that mean that the ingredients used are very nice or it means that the way they cook it is very nice or it means that the cook is a good cook.

7.1.12 Compare user research with user trial.

With user research, data is collected by obtaining users’ responses to questions. User trial data is collected by observing users’ behaviour.

7.1.13 Define expert appraisal.

The reliance on the knowledge and skills of an expert in the operation of the product.

7.1.14 Describe one advantage and one disadvantage of using expert appraisal to collect data.

For example, expert knowledge and advice are gained (compared to a user trial), but the expert may be biased. It may also be difficult to locate an expert. Data is usually qualitative.

Advantages:

  • Experts are experienced in their field of specialty, so they should have the most accurate knowledge.
  • Expert appraisal is inexpensive.
  • It can be both quantitative and qualitative.

Disadvantages:

  • It could be biased as experts tend to look at matters only from their points of view.
  • It may be difficult to locate an expert who is qualified enough.
  • Sometimes it is hard to get data that is relevant.

References

Bulleted list and italicised paragraphs are excerpted from Design Technology: guide. Cardiff Wales, UK: International Baccalaureate Organization, 2007.

Images are clickable links to its location.

Edit - History - Print - Recent Changes - Search
Page last modified on October 08, 2013, at 11:43 PM