Survey Results

Creativity Support Index (CSI)

Results

Figure CSI1 indicates the overall results from the csi and SUS measures, showning that creativity was supported more than usability. There was a near significant correlation between csi and SUS across participants (Pearson’s r(8) = 0.62, p = 0.054). While both represent scores on a 0-100 scale, the two measures should not be considered orthogonal as an assessment of the interface. The specificity of the csi reflects weighted perceptions based on its factor structure, rather than the broader framing of the SUS. Addressing the overall average (unweighted) responses from users (see Figure [CSI2]), the two factors with the highest averages were Collaboration (M (SD) = 7.83 (1.68)) and Enjoyment (M (SD) = 8.21 (1.88)). In particular, the high average score for Enjoyment does not contribute towards the final csi rating for many users, as its weighted value tends towards the mid-lower range. Observing the differences across users in Figure [CSI2], Exploration and Expressiveness are the only two factors to feature in all users WAFS responses. This result indicates individual variation in user perception of their needs when working with sound and music, but also the importance of being able to work on the content at hand effectively; exploring ideas with suitable levels of sonic detail and immediacy of control. These results are coherent with feedback in the Thematic Analysis, where users discussed exploration, immersion, engagement, tool transparency and expressiveness.

AttrakDiff

Results

Results indicate high factor values for Attractiveness (M (SD) = 5.48 (1.01)), the Hedonic Quality of Identity (M (SD) = 5.39 (1.18)), and the Hedonic Quality of Stimulation (M (SD) = 5.91 (0.88)), but only a moderately positive response to Pragmatic Quality (M (SD) = 4.01 (1.49)). The results indicate a predominantly positive attitude towards the Invoke system as a tool for spatial audio. In particular, the Hedonic Stimulation items show high mean scores for differentials such as ordinary/novel (M (SD) = 6.25 (0.75)), conservative/innovative ( M (SD) = 6.17 (0.58)), and dull/captivating (M (SD) = 5.83 (0.83)). Echoing the usability concerns discussed throughout analyses, it can be seen that the system suffers from issues related to confusion, complexity, practicality, and general usability. Despite these usability issues, the other more hedonic qualities indicate positive attitudes.

Collaborative Sonic Interaction Questionnaire Likert Responses

In addition to other survey data, a series of Likert scales were designed to gather data on a variety of factors. These items and results were not included for brevity. Adapting usability questionnaires, mutual engagement questionnaires, and group learning systems, the CoSIQ aims to gather broad feedback using open response and likert items, grouped into factors.

Unless specified otherwise, Likert ranges are: 0 N/A; 1 I do not agree at all; 2 Do not agree; 3 Neither agree or disagree; 4 Agree; 5 I fully agree. Each group contains a single open response item (in italics), to promote subjective impressions, these are listed on another page, and are dealt with in the Thematic Analysis section.

Exploration and co-construction of meaning

Questions

• CQ04_01 = We built on each others’ ideas;

• CQ04_02 = When things were unclear we worked together to resolve it;

• CQ04_03 = I could understand what my partner was trying to do musically;

• CQ04_04 = We learned from each other;

• CQ04_05 = We did not understand our actions towards spatialising music.

Results

Likert responses in this category showed a broad trend that users’ felt that exploration was a positive feature, with most items mean being placed in an agreement with statements (CQ04_04).

Collaboration

Questions

• CQ02_01 = The music made was a shared product;

• CQ02_03 = I felt part of a collaborative process;

• CQ02_04 = I worked mostly on my own;

• CQ02_05 = The other person ignored my contributions.

Results

When looked at together all four items make a poor scale. But further reliability analysis indicated that two groups of Likert items had moderate to high agreement, grouping first two and last two Likert questions.

Satisfaction

Questions

• CQ03_01 = I like the music composition we made;

• CQ03_02 = I enjoyed making the music composition;

• CQ03_03 = I felt part of a creative process.

Results

Likert responses showed a consensus that they enjoyed making the music composition (CQ03_02), and that it felt like a creative process (CQ03_03). Appreciation of the output of collaboration was ambiguous (CQ03_01).

Flow

Questions

• CQ06_01 = It was challenging;

• CQ06_02 = I felt distracted;

• CQ06_03 = I enjoyed myself;

• CQ06_04 = I lost track of time;

• CQ06_05 = It felt like playing a game;

• CQ06_06 = I was bored.

Results

The open-question for flow was a poor question, as it elicited non-explanatory responses. In general, items indicated a capacity for the system to support aspects of flow, with most users’ agreeing or strongly agreeing that they: enjoyed the situation (CQ06_03), lost track of time (CQ06_04), and were not bored (CQ06_06). An important trend was that most users did not feel distracted (CQ06_02), this could be said to detract from a state of flow .

Social Presence

Questions

Variable code questions relate to nmmsp items in Appendix Table [app:measures:nmmsp].

• Co-presence - Mutual awareness

  • CQ09_01 = NM02 01 I hardly noticed the other musician.

  • CQ09_02 = NM02 02 The other musician didn’t notice me in the room.

  • CQ09_03 = NM02 03 I was often aware of the other musician in the environment.

  • CQ09_04 = NM02 04 The other musician was often aware of me in the room.

• Psychological Involvement - Mutual Understanding

  • CQ10_01 = NM05 01 My opinions were clear to my partner.

  • CQ10_02 = NM05 02 The opinions of the my partner were clear to me.

  • CQ10_03 = NM05 03 My thoughts were clear to my partner.

  • CQ10_04 = NM05 04 My partner’s thoughts were clear to me.

  • CQ10_05 = NM05 05 My partner understood what I meant.

  • CQ10_06 = NM05 06 I understood what my partner meant.

• Behavioural Engagement - Interdependence

  • CQ11_01 = NM06 01 My actions were dependent on the my partner’s actions.

  • CQ11_02 = NM06 02 My partner’s actions were dependent on my actions.

  • CQ11_03 = NM06 03 My behaviour was in direct response to my partner’s behaviour.

  • CQ11_04 = NM06 04 The behaviour of my partner was I direct response to my behaviour.

  • CQ11_05 = NM06 05 What my partner did affected what I did.

  • CQ11_06 = NM06 06 What I did affected what my partner did.

• Behavioural Engagement - Mutual Assistance

  • CQ12_01 = NM07 01 My partner did not help me very much.

  • CQ12_02 = NM07 02 I did not help my partner very much.

  • CQ12_03 = NM07 03 My partner worked with me to complete the task.

  • CQ12_04 = NM07 04 I worked with my partner to complete the task.

• Behavioural Engagement - Dependent action

  • CQ13_01 = NM08 01 My partner could not act without me.

  • CQ13_02 = NM08 02 I could not act without my partner.

Results

Within the Likert responses to social presence items, based on factors, the following trends were observed:

• Co-presence: Mutual awareness - high averages with small ranges, indicating good intersubjective awareness when using the system.

• Psychological Involvement: Mutual Understanding - high averages with small ranges, indicating good awareness and comprehension of each others opinions.

• Behavioural Engagement: Interdependence - broad ranges and moderate averages, indicating ambiguous perception of action interdependence when using the system.

• Behavioural Engagement: Mutual Assistance - broad ranges and moderate to high averages, indicating varied relationships to assistance across users.

• Behavioural Engagement: Dependent action - low averages with small ranges, indicating a sense of independence when doing the task.

Sonic Interaction

Questions

• CQ08_01 = I could explore sound using body gestures;

• CQ08_02 = I did not feel in control of the sound;

• CQ08_03 = I was connected to the sound changes through my actions;

• CQ08_04 = I could relate actions to specific sonic effects;

• CQ08_05 = I could plan sonic events using the system;

• CQ08_06 = I was able to make the changes I wanted to the composition;

• CQ08_07 = I was really engaged in the music making process.

Results

Three items within the likert responses showed positive agreement (CQ08_02, CQ08_03, CQ08_04, CQ08_07), suggesting that users’ perceived a level of understanding in sound action and engagement in process. More ambiguous responses were observed for the ability to plan and control sound actions with a level of detail (CQ08_05, CQ08_06). Also the level of embodiment of gesture and sound had ambiguous results (CQ08_01).

System Usability Scale (SUS)

Questions

The answers are formatted using a rating scale (Likert scale) of five levels (Strongly disagree, Disagree, Neutral, Agree, Strongly agree) that range from 1 to 5.

• CQ07_01 = I think that I would like to use this system frequently;

• CQ07_02 = I found the system unnecessarily complex;

• CQ07_03 = I thought the system was easy to use;

• CQ07_04 = I think that I would need the support of a technical person to be able to use this system;

• CQ07_05 = I found the various functions in this system were well integrated;

• CQ07_06 = I thought there was too much inconsistency in this system;

• CQ07_07 = I would imagine that most people would learn to use this system very quickly;

• CQ07_08 = I found the system very cumbersome to use;

• CQ07_09 = I felt very confident using the system;

• CQ07_10 = I needed to learn a lot of things before I could get going with this system.

SUS Calculation

To calculate the SUS score, first sum the score contributions from each item. Each item’s score contribution will range from 0 to 4. For items 1,3,5,7 and 9, the score contribution is the scale position minus 1. For items 2,4,6,8 and 10, the contribution is 5 minus the scale position. Multiply the sum of the scores by 2.5 to obtain the overall value of SUS. SUS scores have a range of 0 to 100, where the higher the value, the higher level of usability.

Results

Given that a standardized test was used for usability (SUS), the results are summarised in the general system score, fig. [fig:invoke:csisus], individual scores can be seen in [fig:invoke:cq07].