top of page

Wattpad is a multi-platform application that focuses on three components: reading, writing, and community. For the reading component, users gets to explore everything from professionally printed books to user-generated material varying across various genres. For the writing component, the platform allows users publish their own work to the platform or share it through social media. Additionally, for the community component, Wattpad allows users to create and post to threads created by other users. 

Executive summary:

For this project, I am doing it for my ISM 360 class, User-Centered Evaluation, along with 3 other classmates. The purpose of this usability testing is to evaluate the platform of the website, Wattpad. Before doing the usability testing with participants, we first each did our own evaluation of the website, then came together to organize our findings and did a user research. For the user research, we were to find the website's target users and interview them about the website. After the user research, we planned our usability testing based on the results we got from the research and came up with task scenarios where we can better understand the features of  the website. Our goal was to find the strengths and weakness of the website and to provide recommendations based on our findings to help improve the user experience of the website as a whole.

Evaluation methods:

The first evaluation method I used was the Heuristic Evaluation. Heuristic evaluation is a method that is used by usability experts to find usability problems in an interface with heuristics that they agreed on. The heuristics that I used to evaluate the website is based on Nielsen's 10 Heuristics:

1. Visibility of system status 

3. User control and freedom

5. Recognition rather than recall

7. Aesthetic and minimal design

9. Help users recognize, diagnose, recover         from errors

2. Match between system and real world 

4. Consistency and standards

6. flexibility and efficiency of use

8. Error prevention

10. Help and documentation

The heuristic evaluation is based on the overall usage of the website, Wattpad. Since the website already exists in the world, the evaluation is based more on if the heuristics were applied in the website. After finding some issues throughout the website, they were then rated based on a severity rating scale from 0 to 4:

0 = not a problem at all

1 = cosmetic problem only: doesn't need to be fixed unless extra time is available

2 = minor usability problem: fixing this should be given low priority

3 = major usability problem: fixing this should be given high priority

4 = usability catastrophe: MUST fix this before product can be released

After doing my own heuristic evaluations, I combined with other classmate's individual heuristic evaluation. Through his evaluation, we got to see the problems that we found and his way of evaluating the website. 

​

The second evaluation method I used was, the Cognitive Walkthrough. A cognitive walkthrough is a method that is used to imagine someone’s thoughts and actions when they are using the application for the first time. The walkthrough is done by designers, people on the team, and users that aren’t the end user of the application. It is down by creating a scenario where it gives a context of who would be using the application and a list of tasks for the walkthrough. The scenario that I came up with for the walkthrough is: At a coffee shop on campus, Jess is on Wattpad looking for a book to read. She wants to get lost in a book based on her environment right now, a coffee shop. If she found one, she wants to find more books to read about what can happen in a coffee shop. This user has a Wattpad account but haven’t explore much of the site. For my cognitive walkthrough, I came up with seven tasks to evaluate and answered four questions for each task. At the end of each task, I tried to add any comment or issues that I saw and any recommendations for the issues. 

Tasks:
Questions:

  1. Log in to Wattpad account

  2. Search "college #coffee

  3. Click on "Finding Love in a Coffee Shop"

  4. Click on "Read"

  5. Click on "+" and add to library

  6. Click on "Table of contents" and go back to book's main page

  7. Click on "coffeeshop" tag

  1. Will the user realistically be trying to do this action?

  2. Is the control for the action visible?

  3. Is there a strong link between the control and the action?

  4. Is feedback appropriate?

Summary of Positive Findings:

After doing both evaluations, my overall experience in using Wattpad was easy and straightforward. The website wasn't that hard to navigate through and the link between the control and the action was consistent and got me to where I intended to go. The website was able to make actions visible as well as buttons that did what I thought it would do. there wasn't too much information on a page and the consistency for each page is in a form layout.

Summary of Usability Problems:

From both the heuristic evaluation and cognitive walkthrough, there were problems that were not severe to problems that needs immediate attention. For the heuristic evaluation, I combined the problems that I found along with problems that another classmate found into a prioritized list os findings. 

User Research:

After our individual evaluations of the website, we came together to come up with interview questions to find the website's target users. When going about the interview process, we individually sought out multiple participants. To find the participants, we each interviewed people based on our screener questions to see if they were an appropriate fit for our user research. We each ended up interviewing 2 to 3 participants each with chosen participants that are current users, have used Wattpad before, or have interest using a site similar to Wattpad. After the screener is done and we have the right participants, we read to them a consent form. The consent form informed all participants of the overview and purpose of the research, the risks, stresses, and discomforts that they might encounter, other information about the research, and the insurance that they could quit the interview process at any time. 

​

When we found participants that fit our research and they signed the consent form, we moved on to the interview. We asked the questions that got them to warm up to us before moving on to questions that is more specific to the research. For the deep focus questions of the interview, since we have current and past users to new users of the website, we had two different categories based on their level of expertise. From these specific questions for the two level if users, we got to see what each group liked and disliked about the website, but also how they navigate through the website.

​

After all the interviews, we compiled the user data into an affinity diagram to look for patterns in out interview data. The purpose of this step was to find patterns and themes across our datasets. We categorized our findings into five sections and sorted all our individual data accordingly on post-it notes. The five categories were:

1. Positive findings

2. Usage

3. Stressors

4. Changes

5. Other applications

To further synthesize our findings, we created four personas to project and craft a simplistic way to share our findings. Personas are created to give a realistic representation of who our audiences are and comes from the user research that was conducted brier to creating a persona. In our personas, we focus on who our audiences are, positives they found while using the website, frustrations they had, other apps they use that is similar to Wattpad, their interests, and user experience with Wattpad. 

​

​

​

​

​

​

​

​

​

​

​

​

​

With the personas, we were also able to come up with four scenarios that goes with the personas. Scenarios give context as to why and how the user uses the site. The type of scenario we used was elaborated scenarios where it gives user characteristics to have a deeper understanding on who the user is and why they use Wattpad the way they do.

Personas:
Scenarios:

  1. While Jessica was in the library before class, she went on Wattpad to look for a book to pass time. She immediately searched for fiction books to read. She noticed how after the search, she could narrow her search down with the suggested refined by tags. She read through the first chapter and noticed that it was time for her class. She saved the book by logging in to her account. After class, she was able to find the book in her library on her phone for her train ride home.

  2. Larry Davis spent a lot of time reviewing his students' writing works. It took countless hours to critique the writing pieces and explain the reviews to students. Larry wanted to be able ti review written content on the same page he was reading and share it immediately with the author. When Larry told his students to use Wattpad, they were able to post written works at no cost and directly give viewable access to their teacher. Soon enough, Larry was reading his students' works from his phone during such and on the train commute back to his home. Larry even shared his students' work with his family by using a Facebook hyperlink. Now, all writing courses in Larry's school recommend Wattpad for professional reviewing.

  3. Ashley is a teenager that loves reading. She is always in the hunt for upcoming Indie authors that self-publish content. Being in high school, she can't afford to pay for books, so she seeks websites that offer her free-to-read stories. She found Wattpad through a friend and has used it frequently on her iPad or MacBook, depending on if she is at home or on-the-go. She likes how she can start a book and continue reading on her iPad on the bus. However, when using the iPad, she often finds it difficult to begin reading a book as the device has no hover states. However, once reading the book, it's east for her to toggle through pages or skip to different points in the book. She likes how she can follow authors on the site, but has found the author pages to be inconsistent and laid out poorly. Overall, Ashley will continue to use the website for her daily reading of self-published pieces. 

  4. Jessica is the type of person who is always between class, work, and friends. However, She has a passion for books and rarely finds time for herself to read. 

Usability Testing:

The purpose of creating a Usability Testing Plan for Wattpad is, through user experience on the website, discover through the More program how the user interacts with certain features by looking at the video and seeing which icons, buttons, or content that click threw. By tracking the users expression through the More program, having the users complete a series of tasks, and analyzing these two components together it will give us researchers a better perspective into the underlining programs regarding usability.

​

Usability testing is when a group of researchers watch a user perform a list of tasks. The researchers create a task list and is even to the user that is testing the application. The user is examined through the recording along with their actions. The user should be given little to guidance during the testing process so that the research team gets a real look at how difficult tasks are to perform from a user perspective. 

Target users:

The personas above included users that were experienced and not experienced with Wattpad. These users were all above eighteen years old. When developing personas, the team believed that users needed specific attributes. Those attributes would include being active on the internet, involved in reading or writing in any manner, and at least in high school. The high school and above criteria was needed since some material on Wattpad is age sensitive. 

​

Many of the recruits will be the same participants from the user research interviews. Since not all participants were available to come in for the usability testing, any new participants will be screened to see if they best fit the usability test. Our screener was based mostly around how often the participants have read over the past year and what website they use and are familiar with. 

Screener:

  1. Please select your gender

            a. Female​​

            b. Male

  2.  Please tell us your age ___

  3.  Have you used any of the following websites in the past year?

             - Google eBookstore

             - Wattpad

             - Project Gutenberg

             - GoodReads

             - WhichBook

             - Reader's digest

             - Amazon

             - ReadPoint

             - BookFinder

        If yes, then which websites are you familiar with?

  4.  Do you have trouble finding books for school, work, or personal interests?

  5.  Which of the following applies to you within the last year?

             - I bought a book that I did not even use

             - A book I was required to buy was free online

             - I could not find a book on the topic I was looking for

             - No one would evaluate my written works

             - I struggled to share some written work

Test environment:

For our research, we held the usability testing in a room where it's specific for research study. In this room, there were two section, the observer side and the user study side. In the observer's side of the room, there is a computer that is connected to the one that the participant used. On that computer, we researchers were able to see their screen, facial expression, and take notes of the recording using More. As for the participant's side of the room, they would be seated in front of a computer where there is a camera to record their facial expression and a moderator seated with them. There were no interruptions during the testings due to the room being closed and no window or blinds was open. The participants were aware that there were other people in the room beside the moderator but would not be able to see them due to the one-way mirror separating them.

Procedure:

After we got the right participants and the room booked for the usability testing, we started off the testing by making sure that the More program and computers in the room were working. Once we knew that we got everything set up, we made sure that the moderator had the documents they needed to do the testing. Once we got participants settled in, for each testing, the moderator read off the script where it states what they need to say to the participant and how much time they should spent on each section. Before the participants did any of the task scenarios, they were first explained to about the purpose of the usability test and given a consent form to sign. The moderator explained each section in the consent form and have the participants sign it before the testing begins. Once the participant was ready to start, the moderator read off an example task to get the participants warm up and be comfortable with talking out loud so we could follow their train of thought and what they saw and did. This way we can better understand why the task was done that way and if there were any confusion or questions that they have. After the example task, the moderator then would give the participant a sheet of paper that contains four tasks that they are to perform and additional information they need to perform one of the task. In addition to the tasks being on a sheet of paper, the moderator also read out loud each task that was needed to perform. Once all our tasks were done, the participants were asked to complete a post-questionnaire that helps us understand their view of the interaction, their impression, and any recommendations that they have for Wattpad.

Task list:

When constructing user testing, the participants will perform a series of four short tasks that are all main features of Wattpad. 

You're a huge George Orwell fan and want to find your favorite book, 1984. Find the book and go to page one.

For the first task, the user will be asked to test out the search feature. We will ask the user to find a pre-selected book and start "reading" the book, with the intent that they will be able to successfully locate and click on the button "read" to complete the task. This will be insightful for us as researchers because browsing and reading a book is a main feature that Wattpad has. The importance of usability on this feature is substantial since it is often used by existing users.

You want to create a story with the details provided to you labeled "Task 2 User Information". Create the story and publish it.

The second task that the user will perform is creating and publishing a story with pre-written content. This is testing out another main feature on the website, for the task, the user will be instructed to create a story and input the information that our research group gives to them. The user will have successfully completed the task if they click the "publish" button.

You want to become more involve in the Wattpad community by posting in the newest club discussion. Find the newest club discussion and post "Hi".

The third task the user will perform is locate a forum on the website under the community tab and post into the forum messages with the account that we provide for the users. This is a used function on the website, however, not as important as the first two tasks. By doing usability testing on this feature, it will give us insight to if this task is underused due to difficulty in performing the task or if it is too embedded and doesn't have enough visibility on the website's interface. The user will have successfully completed the task when they have clicked "submit" and their post shows up in the forum.

You want to message "Check out this website http://mywafflewaffle.com/" to user24782168 on Wattpad. Send user24782168 this message.

Finally, the user will test out the messenger function, which is another underused function. The task entails that the user wants to message an existing account owned by the research group. The user will be instructed to message said user a link that will be also provided, the user will have successfully completed the task when they click send and the message is sent to the given user account.

Evaluation measures:

Every user observed in the Wattpad usability test was evaluated by specific measures. Evaluation measures were present in every task the user had to complete. Markers helped record important information, but hearing the participant vocalize actions helped the most. With the participant vocalization, their thought process became easier to understand how they would approach each task. Usability metrics were used at different portions of the usability test. Effectiveness was observed to assess how accurate a participant can complete a specific task. Efficiency was observed to see in what manner and to what expense was needed for each task to be completed. Satisfaction was recorded to understand the comfort level of each participant completing the tasks.

​

All four tasks were recorded for time and it helps evaluate the efficiency of completing the task. The research team made timestamps of each participant's actions and compared them with one another. A few participants completed tasks quicker than others and milestones were achieved at different times. Effectiveness was also observed in each task by completion rate. Some participants could not complete a task, Comparing the amount of tasks completed by each participant was done once the usability testing was complete. The number of errors made also helped understand effectiveness. Satisfaction was measured through the post-questionnaire, Each participants went through a questionnaire and it did not matter if they had incomplete tasks. The questionnaire provided insights on the task difficulty for each participants and also uncovered the participant's impression of the overall ease of use of the website. With usability metrics, it became easier to observe and quantify the usability of Wattpad. Three of the core elements of usability are used to evaluate each participant's actions. With these three core elements, it became easy to understand the ease and learnability of Wattpad. 

Main Findings and recommendations:

From the main findings, we came up with some recommendations to help fix the issues.​​

  • Easier to find user profiles and the section easier to find

  • Fill in white space in menu bars

  • Make sure only have to select book genre once when creating a book

  • Modernize layout of clubs and discussions

  • Make discussions easier to reply to 

  • Make discussions be in the order of newest to oldest posts

Conclusion:

After conducting an individual evaluation, user research, and usability testing for Wattpad, we got to better understand the target users and know the usability of the website. Like any other websites and applications out in the world now, there are still a lot of constant fixing and researching that is in a cycle where it have to satisfy the users need. After doing this usability testing, there were some changes we would do if we have a chance to do another. We would like to work with a less of a time constraint for some parts of the research so that we can interview users and do more testing with participants. When it comes to usability testing, we would like to have multiple moderators in the room to create more of a conversational atmosphere with no guidance so some testers are more comfortable with speaking out loud about their process. One last thing that we would change is finding a better mix of experience and inexperienced Wattpad users to get a balanced view in both the interviews and the usability testing. 

©2019 by Linh Huynh. Proudly created with Wix.com

bottom of page