Exploring the user’s preferences of different adaptation policies in adaptive menu design

Adaptive User Interfaces (AUIs) have been developed to improve the usability of products by adapting to the user, the platform, and the environment. However, there is a limited understanding of how different adaptation policies impact personalization and usability in adaptive menus. The present study aimed to investigate the effectiveness of different adaptation policies in adaptive menu design. The study surveyed computer science students at Kathmandu University and conducted a usability study to gather data. The results of the study showed that a majority of participants were neutral in their perception of the ease of use of the websites they regularly visit, but a strong majority (83%) indicated a preference for personalized menu options. Personalization was found to be a key factor in the effectiveness of adaptive menus. Participants valued the ability of adaptive menus to tailor their options based on their specific needs or preferences. The findings of this study provide insight into users’ preferences for adaptation policies in adaptive menus and suggest that a recency-frequency-based menu is most effective in meeting users’ needs. Similarly, findings also suggest users’ preference for adaptation policy also changes based on the context of use. Future research could further investigate the effectiveness of different adaptation policies in different contexts of use.

Introduction

The use of software applications has grown significantly in recent years, with an increasing number and variety of web and mobile applications available. A key factor in the success of these digital businesses is the User Experience (UX) and the User Interface (UI) design is central to this. However, the increasing complexity of UI can make it difficult for users, particularly those who are new to using the product. To address this issue and meet the needs of individual users, Adaptive User Interfaces (AUIs) have been developed. Adaptive User Interface (AUI) is a subset of Intelligent User Interfaces (IUIs), which combine elements of Artificial Intelligence (AI), software engineering, Human-Computer Interaction (HCI), and other disciplines. The goal of AUIs is to improve the usability of a product by adapting to the user, the platform, and the environment [1,2]. This includes adapting to the user’s usage patterns, the type of device or screen size, and the specific use case. Responsive design is one technology that is already being used to make UI more adaptable to different screen sizes.

Menus are a common UI element in web and mobile applications, and researchers have focused on both adaptive and adaptable menus to improve navigation and reduce selection time [3]. Adaptive menus are managed by the system and adapt to the user’s usage patterns, while adaptable menus are customized by the user based on their preferences. As a self-customization menu seems only suitable for technically proficient users, adaptive menus are the way forward. The effectiveness of adaptive menus is determined by two main factors: the adaptation style, which specifies how the adaptive menu items should be displayed, and the adaptation policy, which specifies which menu items should be displayed. While there has been a significant amount of research on adaptation styles, there has been relatively little research on adaptation policies [4].

This research aims to identify users’ preferred adaptation policy and examine how it may vary based on the context of use. There is a limited understanding of how different adaptation policies impact personalization and usability in adaptive menus. It is not clear which policy is most effective in meeting user needs. The motivation behind this study is to investigate the potential of adaptive menus to improve the user experience, increase user satisfaction and engagement with the menu system, and allow users to more quickly and easily find the options they are looking for more. The research will be conducted by conducting user studies and providing insights and recommendations.

In upcoming sections, this paper presents a literature review on adaptive menus and their styles and policies, methodology, results of a study on user preferences, discussion of implications and contribution to the field, and summary of key findings and potential future work.

Literature review

Web menus and navigation play a crucial role in user experience (UX) design in web development. They provide a structured way for users to access and navigate the content and features of a website and serve important functions such as helping users orient themselves and understand the website’s structure, enabling clear navigation, and reducing cognitive load [5,6]. Navigation history, which refers to the record of pages or content a user has visited on a website, also serves important functions such as allowing users to retrace their steps and resume tasks. To improve usability and user experience, web menus can be made more usable by providing personalized adaptive menus [7].

Menus are a common UI element in software and as the number of menu items increases, the importance of menu design also increases [8]. Gathering specific requirements of each user can be time-consuming, so UI designers often gather requirements from a small group of featured users [9]. Adaptive menus change and adapt to the needs of users to make it easier for them to complete tasks. Studies have shown that adaptive user interfaces (AUIs) can be effective in different contexts, with screen space and adaptation techniques affecting accuracy [10,11]. Research has also found that the frequency of adaptation greatly affects the relative importance that users place on the costs and benefits of adaptation [12]. There has been much research on improving the usability of menus, including the use of adaptive menus to reduce menu selection time [13]. Adaptive menus can be decomposed into two components: the adaptive style and the adaptation policy. These two components determine the performance of an adaptive menu Figure 1.

Figure 1: Framework for Adaptive Menus [20].

Adaptive menus adjust the display of menu items based on the user’s history and predicted preferences to reduce visual search time. Different forms of adaptation styles include spatial, graphical, and temporal forms. Spatial adaptation involves changing the position of the adapted item, graphical adaptation involves changing the visual attributes of the item such as color or font size, and temporal adaptation involves changing the time at which items are displayed. Sears and Shneiderman introduced the concept of “split menus” which divide frequently selected items from less frequently used items to improve performance [8]. However, this design may not be suitable for small screens, cloud menus may be a better option [14]. Graphical adaptation can also be used in the form of highlighting, bolding, coloring, underlining, or changing the font size [15-17]. Users tend to prefer menus that maintain spatial stability and dislike menus with unusual shapes or color-changing elements [18]. While ephemeral adaptation, which employs a temporal form, was found to be faster when prediction accuracy was high, users tend to prefer other forms of adaptation [19].

Adaptation policies, which determine which menu items should be displayed, can be based on factors such as item click frequency, recency, and page visit duration [20]. These policies, also known as prediction algorithms, help determine the usefulness of each menu item for a given user. Self-Adapting Menus (SAM) is a framework that allows for the exploration of various adaptation policies and styles in adaptive web menus [20]. However, there is currently no consensus on which combination of policies and styles is the most effective. This research aims to investigate users’ preferences for different adaptation policies and how these preferences may vary based on the context of use.

Methodology (Figure 2)

Figure 2: Various stages of study.

Development of prototype

In this study, five prototypes were developed to investigate user preferences for adaptive menu policies. Four adaptive menu prototypes were developed with different policies including recency-based, frequency-based, recency-frequency-based, and serial position curves. A traditional menu served as a control condition with no adaptation policy applied. These prototypes were used as data collection apparatus. The recency-based prototype prioritizes recently clicked items, the frequency-based prototype prioritizes the most frequently clicked items, the recency-frequency-based prototype considers both recency and frequency, and the serial position curve prototype considers recency, frequency, and the order in which items were clicked. The design of these prototypes followed best practices for menu design such as maintaining spatial stability and avoiding unusual designs [18]. The prototypes were developed using JavaScript and DOM manipulation and user data was stored locally. The adaptive menu prototypes calculated scores for each menu item based on the different policies and rearranged the menu items accordingly.

• Recency-based: In the recency-based prototype, the menu item that is being clicked recently at first (left-most part of adaptive menu section).
• Frequency-based: In the frequency-based prototype, the menu items that have been clicked most have the highest score, and menu items with the three highest scores are displayed in the adaptive menu section.
• Recency-frequency-based:In this prototype, the overall score of each menu item is influenced by click recency, if the menu item has been clicked recently then its frequency score is increased by 50%.

Score = Frequency * Recency (Where recency = 1.5)

• Serial Position Curve (SPC): In this prototype, the overall score of the menu item is influenced by both primary and recency. If the menu item is either clicked at the start of the session or recently the overall frequency score gets increased.

Score = Frequency * Recency * Primacy (Where recency = 1.3, primacy = 1.2)

The coded names for each adaptive menu are as follows:

Menu 1: Recency-based menu

Menu 2: Frequency-based menu

Menu 3: Recency-frequency-based menu

Menu 4: Serial Position Curve (SPC) menu

Data collection

The usability study was conducted with 18 participants to gather data on user preferences and behaviors in personalized menus. The age of participants was between 18 and 30, and most of them were students studying computer engineering or computer science at Kathmandu University. The participants had medium to a high level of tech proficiency, and frequently visited websites such as nagarik.com, ekantipur.com, daraz.com.np, and merojob.com. The study was conducted to understand the preferences and behaviors of users in personalized menus and identify potential issues or improvements that could be made to the prototypes. The usability testing was conducted in the Digital Learning Research Lab at Kathmandu University, moderated by the researcher, and using the think-aloud method to gather feedback and observations in real-time [21]. Participants were asked to complete a comparison task using the adaptive menu prototypes, which provided a more in-depth understanding of user preferences and behaviors in adaptive menus. Following the usability testing, a semi-structured follow-up interview was conducted to derive more detailed insights and feedback from the participants.

To understand users’ preferences for adaptive menu adaptation policies in different contexts of use, a questionnaire was developed based on the findings of a usability study and research questions. The data was collected using an online Google Form and mostly consisted of students from computer-related fields at Kathmandu University. A total of 86 responses were collected. Participants were provided with information on the adaptive menu and a link to a recency-frequency-based prototype to provide context and familiarity with the personalized menu concept. The questionnaire consisted of 13 questions, 12 of which were close-ended, covering background and demographics, preferences for adaptive menus versus traditional menus, and preferences based on the context of use. A pilot test was conducted with 3 users to test the clarity of the questions and improve language. The sample size of 86 participants was deemed sufficient for the qualitative research, aiming to identify adaptation policy preferences based on the context of use.

Data analysis

In this study, the analytic strategy is to compare and analyze data from the usability study and questionnaire. The collected data is organized into different themes using thematic analysis, a method of qualitative data analysis that involves recognizing, categorizing, and analyzing recurring themes in data to determine their underlying meaning and significance [22]. This allows for the identification of patterns and relationships in data, as well as trends and variations that can help in answering research questions. The data from the user interviews were coded according to themes and categories and analyzed to identify patterns and trends. The themes used in this study are various adaptation policy names used in different prototypes, such as recency-based, frequency-based, recency-frequency-based, serial position curve, and context of use. The coding process assists in tagging only relevant responses to the research questions.

Results

This study targeted computer science students at Kathmandu University and received responses from a total of 86 participants. When asked about the types of websites they regularly use (excluding social media), a significant portion of respondents reported using e-commerce sites like Daraz and Hamrobazzar (32.2%), news portals like Ekantipur and Nagarik (31%) and educational websites like Coursera and W3School (24.1%), as shown in Figure 3. In terms of frequency of use, 19.5% of participants reported using these websites more than three times per day, 35.6% used them one to two times per day, and 36.8% used them one to three times per week (Figure 4). This data suggests that a majority of users (55%) access these types of websites at least once per day. Overall, the results of this study provide insight into the common types of websites accessed by participants, as well as the frequency with which they use these websites.

Figure 3: Types of websites participants use (excluding social media).

Figure 4: Frequency of website uses.

During the survey, participants were asked about their perceptions of the ease of use of the websites they regularly visit. The majority of participants (55%) were neutral in their responses, while 31% indicated that they were satisfied with the menu options provided by these websites. Only 14% of participants were unsatisfied with the menu options (Figure 5). However, when asked about their preference for personalized menu options, a strong majority of participants (83%) indicated that they would prefer to have personalized options. Of these, 42.5% were strongly in favor of personalization, 13% were neutral and 4% were against it (Figure 6).

Figure 5: Users’ rating of the websites based on ease of use.

Figure 6: Users’ view towards menu personalization.

In terms of personalization, during the usability study, participants preferred the adaptive menus over the static menu because the adaptive menus provided a sense of personalization. Participants valued the ability of the adaptive menus to tailor their options based on the user’s specific needs or preferences.

“I preferred any of the adaptive menus over the static menu because they provided some kind of personalization.” (Orginal quote in Nepali)

On the other hand, participants suggested that adaptive menus can be less useful if the options they display are not relevant or helpful to the user. In these situations, participants felt that a static menu might be a better choice, as it would not add unnecessary complexity to the user experience. Instead, the static menu would simply provide a set of fixed options for the user to choose from.

“If the displayed options in the adaptive menu are not useful, it just adds complexity, In such case traditional menu would be ideal” (Orginal quote in Nepali)

Adaptation policy preference

The purpose of this research was to understand users’ preferences for adaptation policies in adaptive menus. To determine these preferences, user feedback was collected from interviews, as well as a usability study was conducted in which participants were asked to rank different adaptive menu prototypes based on their level of preference. The results of this study, including the average ranks of each menu and the number of participants who ranked each menu as their top preference, are presented in Table 1. In addition to these rankings, this section also includes feedback and comments on the adaptive menus provided by participants during interviews.

Recency-based menu: The recency-Based menu was preferred by participants due to the recent relevant menu options that were displayed. Participants in their interviews and feedback cited this preference. For example, participants stated:

“This menu feels more personalized to my needs because it showed options that I had just clicked on.” (Orginal quote in Nepali)

“I liked menu1 because it showed options that I had recently used, which made it easier for me to find what I was looking for.” (Orginal quote in Nepali)

On the other hand, some participants found the changes in the recency-based menu to be too frequent, which reduced its appeal. One participant said:

“The changes in Menu 1 were too frequent for me, I prefer any other menu because the changes were more stable.” (Orginal quote in Nepali)

Overall, the Recency-Based menu was ranked first by 2 participants with a top rank percentile of 11.11%, third by 3 participants, and fourth by 13 participants. The average rank of the recency-based menu was 3.5 making it the least preferred menu among participants.

Frequency-based menu: The frequency-based menu was preferred by some participants due to the less frequent changes to menu options. However, other participants found the frequency-Based menu to be too static and not responsive enough to their usage patterns.

“I preferred this menu because it felt more personalized, even though the changes were less frequent than in the previous menu (recency-based menu).” (Orginal quote in Nepali)

“I prefer the previous menu (recency-based) because the changes in this one are too infrequent for my liking. I like menus that adapt more frequently to my clicks.” (Orginal quote in Nepali)

The frequency-Based menu was ranked first by none of the participants, ranked second by 1 participant, and third by 12 participants. The majority of participants ranked it third. The average rank of the frequency-Based menu was 3.22, which was slightly higher than the average rank of the recency-Based menu.

Recency-frequency-based menu: The recency-frequency-based menu was highly preferred by participants due to its stability and relevant menu options displayed by it. Participants found this menu to have more stability when compared to recency-based and to be more responsive as compared to the frequency-based menu. According to user feedback, the RF menu provided a good balance between personalization and stability.

“This menu is my favorite because it provides a good balance between personalization and stability.” (Orginal quote in Nepali)

“I preferred this menu because the options were more relevant to my recent browsing history, but the changes were not too frequent.” (Orginal quote in Nepali)

Many participants cited this menu as their favorite, with 12 participants ranking it first and 5 ranked it second. The average rank of this menu was 1.56, making it the most preferred menu among participants, with an exclusively top-ranked percentile of 44.44.

Serial Position Curve (SPC) menu: The SPC-based menu was preferred by many participants due to its personalized and relevant options. Some participants found it difficult to distinguish between the recency-frequency-based menu and the SPC menu, but overall both were well-received. 22.22% of participants could not choose one over the other.

“I couldn’t tell much difference between this and previous (recency-frequency-based) menu, but I preferred both of them over the other options.” (Orginal quote in Nepali)

“This menu was one of my favorites because the options felt more personalized, but I couldn’t say for sure if it was any better than Menu 3.” (Orginal quote in Nepali)

Whereas other participants could tell the difference based on the personalized menu option it provided as compared to the recency-frequency-based menu. The SPC menu was ranked first by 8 participants, with 4 of those ranking it as their top choice exclusively. It was also ranked second by 10 participants. The SPC menu was the second most preferred menu among participants, with an average rank of 1.72.

Adaptation policy preference based on context of use

This research also examined users’ preferences for the adaptive menu in different contexts of use. A questionnaire was administered to participants asking whether they thought the personalization provided by the menu would be helpful for various types of websites. Participants could select multiple websites where they felt the feature would be useful. The results showed that 91% of users thought the adaptive menu would be helpful for e-commerce websites, 78% preferred it for news portals, 57% for educational websites, 47% for services/job portals, and the lowest percentage (19%) preferred it for social media. These results are shown in Figure 7.

Figure 7: Percentage of participants who prefer adaptive menus for different types of websites.

Another survey was conducted to determine which factors users consider most useful for personalized menus in different contexts of use. The results of this survey showed that for news portals (such as ekantipur, nagarik, and BBC), 59% of participants preferred menus based on frequency (i.e., the most frequently clicked or viewed categories), 29% preferred menus based on recency (i.e., recently clicked or viewed categories), and 13% preferred menus based on primacy (i.e., categories clicked at the start of a session) (Figure 8a).

Figure 8a: Preferences for personalized menus on service or job portals.

Figure 8b: Preferences for personalized menus on service or job portals.

Figure 8c: Preferences for personalized menus on e-commerce websites.

Figure 8d: Preferences for personalized menus on educational websites.

Figure 8e: Preferences for personalized menus on social media websites.

For service or job portals (such as merojob, sriyog, and upwork), 52% preferred menus based on frequency, 35% preferred menus based on recency, and 13% preferred menus based on primacy (Figure 8b).

For e-commerce websites (such as amazon, daraz, and hamrobazzar), 61% preferred menus based on recency, 25% preferred menus based on frequency, and 14% preferred menus based on primacy (Figure 8c).

For educational websites (such as Coursera and W3School), 44% preferred menus based on frequency, 33% preferred menus based on primacy, and 23% preferred menus based on recency (Figure 8d).

Finally, for social media websites (such as Facebook and Instagram), 66% preferred menus based on recency, 25% preferred menus based on frequency, and 9% preferred menus based on primacy (Figure 8e). Overall, these results suggest that users have different preferences for personalized menus depending on the type of website they are using.

Discussions

The first aim of this study was to understand users’ preferences for adaptation policies in adaptive menus. The results of the usability study and online survey indicated that the recency-frequency-based menu was the most preferred adaptation menu, followed by the Serial Position Curve (SPC)-based menu. The recency-based and frequency-based menus were ranked lower in terms of preference. Several factors may have contributed to the high ranking of the recency-frequency-based menu. One possible reason is that this menu combines the benefits of both recency and frequency, providing users with a mix of both recently used and frequently used options. This may have made the menu more useful and relevant to users, as it was able to better tailor the options to their specific needs and preferences. Additionally, the SPC-based menu may have been more effective at helping users navigate and explore new content. Users reported that recency-based and frequency-based menus were less effective at providing relevant and useful options over time.

The second aim of this study was to investigate how user preferences for adaptive menu adaptation policies may vary based on the context of use. The results of this study suggest that user preferences for adaptation policies may vary depending on the type of website and the user’s familiarity with the website. Based on results, A recency-frequency-based menu with a low influence of recency is recommended for news portals and job portals, a recency-based or recency-frequency-based menu with higher recency influence for e-commerce and social media websites, and an SPC-based menu for educational websites. These results provide insight into the factors that influence user preferences for adaptive menus and offer guidelines for designing adaptive menus that are effective and user-friendly.

Conclusion and future works

In conclusion, this study has shown that user preferences for different adaptation policies in adaptive menus can vary based on the context of use. The recency-frequency-based and SPC-based menus were found to be the most preferred among participants. However, it is important to note that the context of use plays a crucial role in determining user preferences and these preferences should be taken into account when designing adaptive menus. This study has limitations such as limited user demographics, single-point measurement, and limited context of use. Future research can expand the sample size, test different adaptation policies in different contexts, evaluate the long-term impact of adaptive menus, and develop new adaptation policies for better meeting user needs.

1. Hartmann M. Challenges in Developing User-Adaptive Intelligent User Interfaces. In Proceedings of the 17th Workshop on Adaptivity and User Modeling in Interactive Systems. 2009.
2. Gonçalves T, Rocha A. The development process for intelligent user interfaces: an initial approach. In Proceedings of the XVIII Brazilian Symposium on Software Quality. 2019; 210-215.
3. Gajos KZ, Czerwinski M, Tan DS, Weld DS. Exploring the design space for adaptive graphical user interfaces. In Proceedings of the Working Conference on Advanced Visual Interfaces. New York, NY: ACM. 2006; 201–208.
4. Ehlert PA. Intelligent user interfaces: Introduction and survey. The Delft University of Technology, the Netherlands. 2003.
5. Nielsen J, Tahir M. Homepage usability: 50 websites deconstructed. Indianapolis, IN: New Riders. 2002.
6. Morville P, Rosenfeld L. Information architecture for the world wide web (3rd ed). Sebastopol, CA: O'Reilly Media. 2006.
7. Kolski C, Le Strugeon E. A review of intelligent human-machine interfaces in the light of the arch model. International Journal of Human-Computer Interaction. 1998; 10(3): 193-231.
8. Sears A, Shneiderman B. Split menus: Effectively using selection frequency to organize menus. ACM Transactions on Computer-Human Interaction. 1994; 1(1): 27-51.
9. Findlater L, McGrenere J. A comparison of static, adaptive, and adaptable menus. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. Vienna, Austria. 2004; 89–96.
10. Findlater L, Mcgrenere J. Impact of screen size on performance, awareness, and user satisfaction with adaptive graphical user interfaces. In Proceeding of the Twenty-sixth Annual SIGCHI Conference on Human Factors in Computing Systems. New York, NY: ACM. 2008; 1247–1256.
11. Tsandilas T, Schraefel MC. An empirical assessment of adaptation techniques. CHI ’05 Extended Abstracts on Human Factors in Computing Systems, New York, NY, USA. 2005; 2009–2012.
12. Gajos KZ, Everitt K, Tan DS, Czerwinski M, Weld DS. Predictability and accuracy in adaptive user interfaces. In Proceeding of the Twenty-sixth Annual SIGCHI Conference on Human Factors in Computing Systems. New York, NY: ACM. 2008; 1271–1274.
13. Kleiss JA. Identifying users’ conceptual organization of menu functions in an automotive electronic navigation product. Proceedings of the Human Factors and Ergonomics Society 41st Annual Meeting, 1997; 944-948.
14. Bouzit S, Vanderdonckt J, Calvary G, Chêne D. Cloud Menus, a circular adaptive menu for small screens. IUI, AMC, 2018; 7-11.
15. Antwarg L, Lavie T, Shapira B, Rokach L, Meyer J. Highlighting items as means of adaptive assistance. Behavior & Information Technology. 2013; 32(8): 761–777.
16. Liu W, Bailly G, Howes A. Effects of frequency distribution on linear menu performance. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI '17). Association for Computing Machinery, New York, NY, USA. 2017; 1307-1312.
17. Tanvir E, Cullen J, Irani P, Cockburn A. AAMU: adaptive activation area menus for improving selection in cascading pull-down menus. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '08). Association for Computing Machinery, New York, NY, USA. 2008;1381-1384.
18. Vanderdonckt J, Bouzit S, Calvary G, Chêne D. Exploring a design space of graphical adaptive menus: Normal vs. small screens. ACM Transactions on Interactive Intelligent Systems. 2019; 10:1-4.
19. Findlater L, Moffatt K, McGrenere J, Dawson J. Ephemeral adaptation: The use of gradual onset to improve menu selection performance. In CHI '09: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. New York, NY: ACM. 2009; 1655-1664.
20. Gobert C, Todi K, Bailly G, Oulasvirta A. SAM: A Modular Framework for Self-Adapting Web Menus. In Proceedings of the 24th International Conference on Intelligent User Interfaces. 2019; 481-484.
21. Ericsson KA, Simon HA. Protocol analysis: Verbal reports as data. Cambridge, MA: MIT Press. 1993.
22. Braun V, Clarke V. Using thematic analysis in psychology. Qualitative Research in Psychology. 2006; 3(2): 77-101.