1. Introduction
Synchronized movement (SM) is a unique and complicated human behavior engaging various dimensions of cognition in an orchestrated way (
Basso et al., 2021). The prosocial effects of SM are reported in multiple previous studies (
Hove & Risen, 2009;
Launay et al., 2014;
Valdesolo & Desteno, 2011). Despite past attempts to find the mechanisms behind these effects, the multifaceted nature of SM makes it challenging to study in a controlled experimental design. Confounding factors and ecological validity in such studies are the most hindering obstacles. Different methods of studying synchrony in laboratory settings are typically limited to explicit instructions (for instance, mimicking teammates vs performing movements without paying attention to their movements) or presenting different types of stimuli (for instance, different vs same rhythms) (
Kirschner & Tomasello, 2010;
Tarr et al., 2016). Differentiating the pure prosocial effect of synchrony using these methods has proven to be complicated. Moreover, Tarr et al. have described issues associated with interpreting the results obtained by these methods (
Tarr et al., 2018). One may use the effects of implicit alignment or antagonism in these tasks since the participants are instructed to do the same thing as others (synchronous conditions) or neglect their teammate’s movements (non-synchronous conditions). Additionally, this type of instructed synchrony can produce explicitly shared intentionality between participants, which can partly lead to prosocial effects (
Tarr et al., 2018).
Virtual reality (VR) environments have effectively eliminated confounding factors in studying SM. Accordingly, VR technology is progressively making its way as a methodology in cognitive science (
Faria et al., 2016;
Parsons et al., 2017). More ecologically valid experimental designs for social cognitive studies can be created using VR as VR environments studying SM can be promising since there is no need for populated groups in experiments or therapies (
Parsons et al., 2015). We can also apply robust control to these studies in VR environments. However, creating an immersive and valid VR environment is a demanding task (
Wiltermuth, 2012). Studies involving body motion in a social context face additional complexity to reach an ideal level of plausibility. To study synchrony in VR, we must track human body movements with high temporal and spatial accuracy. Moreover, to make the environment more realistic, participants should have a virtual body (
Waltemate et al., 2018) presence, and emotional response are influenced depending on the specific look of users’ avatars, which varied between (1 in the environment, such that the movements of virtual body parts are matched to participants’ actual movements. The prerequisite of all mentioned tasks is an efficient motion-tracking strategy.
Various solutions and methods for motion tracking have been proposed, each with advantages and disadvantages. Commercial methods, such as the Optitract previously used to study SM are highly accurate but expensive. This method relies on capturing the location of many retroreflective markers on a suit worn by participants, which can be uncomfortable for experimental sessions that last a long duration (
OptiTrack 2022;
Bourdin et al., 2013). Kinect cameras are other devices that can be used for motion tracking (
Bolton et al., 2014). Such cameras are not accurate and fast enough for research purposes, and their precision is also limited when body parts occlude one another. This leads to a failure to capture the full range of motion in some cases (
Caserman et al., 2019).
On the other hand, motion sensing in VR is done by micro-electrical sensors in head-mounted displays (HMD) and controllers. This method is fast and accurate but limited by the number of sensors. Nevertheless, by applying the inverse kinematic (IK) method, the recorded information can even be used to create whole-body motion (
Vox et al., 2021). The IK method uses mathematical models to solve joint parameters needed to place an extremity of a kinematic chain in a particular position and is used extensively in computer animation and robotics. There are IK solutions available to use in VR game designs (for example, the final IK in the Unity Game engine) (
Home - RootMotion, 2022).
A typical VR device uses electrical motion sensors in three locations. The head and two hands belong to the HMD, the left, and the right controllers, respectively. They constantly send the location and rotation of these three points in the space (1 kHz sampling rate). The HMD sensor’s information is vital for adjusting pictures to the head position to create a 360-degree experience. If hand interactions, such as grabbing objects are needed, the information from controllers is also necessary. A typical representation of a player in VR is one head and two unconnected hands (
Caserman et al., 2019;
Parger et al., 2018). However, the mentioned sensors can be used to create a complete upper body motion (two upper limbs, head, neck, and trunk) by solving parameters for other joints computationally via IK. Controlling virtual characters (VCs) by this method can create a sense of body ownership for a participant. However, despite the simplicity and availability of this method, it can bring some issues in experimental settings. Deviating VCs from participants’ authentic movements and limiting sensors to the upper limb, which leaves legs uncontrolled, can negatively affect the sense of embodiment (
Caserman et al., 2019). These issues may disrupt the immersive experience and detach the participant from the VR environment. There is not enough information to gauge the efficacy of this method in studying SM and addressing related issues. Showing the effectiveness of this method for creating a similar VR environment used to study SM can be promising, considering its availability and affordability.
The body of robust evidence supports the positive effect of different types of synchrony in humans. These effects range from psychological to physiological consequences and are not limited to age or context. They are also not limited to healthy individuals (
Hu et al., 2022). One study reported a positive effect of them on empathy in autistic patients (
Koehne et al., 2016). Synchronized activity positively affects helping (
Cirelli et al., 2014), trust (
Launay et al., 2013), liking (
Hove & Risen, 2009), rapport (
Vacharkulksemsuk & Fredrickson, 2012), empathy (
Koehne et al., 2016), conformity (
Paladino et al., 2010), obedience (
Wiltermuth, 2012), attention (
Schilbach et al., 2010), memory (
Miles et al., 2010), the feeling of closeness (
Tarr et al., 2016), the perception of similarity, and the sense of community (
Pearce et al., 2017). Growing studies are investigating the biological basis of these findings, and in this line, some evidence has shown the endorphins’ mediator effect (
Tarr et al., 2016). Neuroimaging data show the engagement of the reward system and other regions (
Kokal et al., 2011). Considering the mentioned effects related to synchrony, devising and evaluating new methods for implementing it can be helpful.
To evaluate the effectiveness of the IK solution for experimental purposes in studying SM, we have used it to develop a simple VR environment. In this environment, a virtual avatar mimics the subject’s movements synchronously or non-synchronously in separate groups. We show that our manipulation of the synchrony affects the reported feeling of synchrony between the two groups.
2. Materials and Methods
Study participants
The participants were recruited from the students of Tehran University in Tehran City, Iran. A total of 92 individuals expressed their interest in taking part in the study. They were interviewed online, and 16 met the exclusion criteria (having psychological illnesses or self-declared movement limitations). The remaining 76 participants (22 females; Mean±SD 25.29±4.72 years) filled out an online pre-screening questionnaire and were informed about the experiment time and location. The participants were randomly assigned to synchronous (SC; n=38) and non-synchronous (NS; n=38) groups. All instructions and questionnaires were identical in both conditions. The Ethics Committee of the Tehran University of Medical Sciences approved the study protocols.
Study procedure
The participants were told that the study was about the effect of movement and lighting on memory in a VR setting. A hypothesis-blind experimenter guided participants through the experiment, and a VR operator helped the participants in the VR room. After signing a consent form, the participants answered a computer-based questionnaire, including demographic questions, a Persian-translated version of the interpersonal reactivity index (
Davis, 1983) questionnaire, and the neuroticism, extraversion, openness-five faction personality inventory test (
Costa & McCrae, 2008). The blind experimenter explained two simple upper body movements and confirmed that the participants learned them thoroughly. The movements were simple periodic arm activities, and we instructed the participants to do them synchronously with a rhythm whenever they were asked in the VR environment.
The operator helped participants wear a head-mounted display (HMD) in the VR lab. After an image quality inspection, the operator started the task and left the VR room. After a few seconds, the virtual body appeared in a first-person perspective. Sets of vocal commands asked the participant to move their hands in different directions to become comfortable with the virtual body. After the familiarization period with the virtual body, a VC appeared in front of the participant. The vocal command asked participants to attend to their teammates and perform movements. After completing each movement one time, the participants were asked to choose any of the two movements and perform it synchronously with the subsequent rhythm. Next, the participants were asked to do the remaining movements when rhythms were played. By letting the participants choose their first move, we tried to avoid constraining them too much, thereby instilling more sense of agency. Each movement was done in 60 s. When the last movement was finished, the participants were informed that the experiment was over and the operator would aid them in removing HMD. During the entire VR time, the information from sensors was recorded. Finally, the participants answered a computer-based post-VR questionnaire, received monetary compensation, and were debriefed. All the steps mentioned above are identical in the two study groups, except the delay time between participants and VC is more pronounced in the NC group (300 frames) than in the SC (20 frames).
VR environment
A simple game scene was designed in the Unity Game engine. This VR environment was a cubical room with matched dimensions to our physical VR lab space. The floor color matched the room’s floor color, and the walls were gray. A red circle was on the VR scene floor to indicate the subject’s position in the scene during the entire task. To avoid the uncanny valley problem, we did not use too realistic textures or additional objects (
Stein & Ohler, 2017).
VR character
A white male VC was created with an avatar-designing app (Adobe Fuse). The VC was then rigged (a process of defining joints and bones for a 3D shape) on Mixamo website. We applied an idle animation to make the character more realistic in the VR environment. Other movements were superimposed on this idle animation. We also added the ability to blink for VC eyes using shape morphing in another software. Blender - A 3D Modeling and Rendering Package; Stichting Blender Foundation, Amsterdam (
Community, 2018).
IK and motion tracking in VR
We used signals from three points in the VR device (left and right controllers and the headset). Rotations and locations in space were collected and assigned as IK targets for the hands and the head of each participant’s virtual body. For IK targets of the VC, we used transformed points. The transformation was done by 180 degrees rotation on the horizontal plane and by adding 180 degrees to rotation around the z-direction of all IK targets. The character was scaled to the same size as the participant. Right and left controllers’ positions were assigned as the right and left hands of the IK targets, respectively, and the headset was used as the IK target’s head. To limit the distance between the participant and the character, we instructed participants to stand on a red dot on the floor. This resulted in a space in VR that corresponded to 2 m in objective measurements. The final result was a character standing face to face with the participant and moving in correlation with the participant’s motion.
Dependent measures
Self-reported questionnaire
We used the Persian translated version of a questionnaire used in previous similar studies to evaluate the following items: The feeling of connectedness, likeability, similarity in personality, degree of synchrony with others, being interested to know others, feeling of success at doing instructions, perceived degree of being followed by avatars and following avatars, feeling that other avatars controlled by real people, feeling that they were in the presence of other people, confidence (remembering movements). All the questions were in the form of computer-based rating slides (based on a 1–100 scale where 1=not at all and 100=completely). Additionally, the inclusion of others in self was asked in the form of visual analog scaling questions.
Social closeness variable
We measured social closeness by rating three observed variables, likeability, similarity in personality, and connectedness. These items are recognized via exploratory factor analysis as observed variables of one latent factor. Factorability of the data was assessed (Kaiser–Meyer–Olkin=0.655, P<0.001). The mean of likeability, similarity in personality, and connectedness was used as a social closeness index (Cronbach α=0.81).
Movement analysis
We used the coordinates of the two hands in three dimensions to extract the Euclidian distance in space. The distance signal was then normalized by dividing it by the maximum value of the signal to cancel the effect of body size variability across participants. The root mean squared (RMS) of the distance signal was used to represent physical activity for each participant. We used dynamic time wrapping to identify the movements. To understand the participants’ efficacy in performing learned movements, we compared the averaged signal of each cycle of any movements with the performance of the blind experimenter. All analyses were performed in MATLAB software, version 2018 (MATLAB and Statistics Toolbox Release 2018a, The MathWorks, Inc., Natick, Massachusetts, United States).
Statistical analysis
The normality of data was checked by the Kolmogorov-Smirnov test. For normally-distributed variables, the t-test was used for hypothesis testing. Hypothesis testing for variables with non-normal distribution was performed by the Mann-Whitney U test. Factor analysis was done by the “factor” analyzer package in Python DS software, version 3.8.
3. Results
Self-reported synchrony
Analysis of participants’ self-reported degree of synchrony (based on a 1–100 scale, where 1=not at all and 100=completely) showed a significant difference in how synchronized participants assessed their movements concerning the VC (Mann-Whitney U test [2-tailed]: n=76, P<0.001), with higher scores in the synchronous condition (median=80.0) compared to the non-synchronous condition (median=65.0) (
Figure 1).
Self-reported social closeness
There was no significant difference between the degree of self-reported social closeness. The Pearson correlation coefficient was computed to assess the linear relationship between self-reported social closeness and self-reported social synchrony (
Figure 2). There was a significant positive correlation between the degree of self-reported social synchrony and self-reported social closeness within the synchronous group (r [36]=0.51, P<0.001) and the non-synchronous group (r [36]=0.47, P <0.01).
Physical activity
The RMS of the hand-distance signal as a proxy of physical activity showed no significant difference between the two groups. There was no significant correlation between the amount of physical activity and reported synchrony within the groups.
4. Discussion
Despite the robust evidence supporting the prosocial effects of synchrony in humans, our knowledge about its underlying mechanisms and applications is limited. Researchers need to use more reliable and accessible methods to answer these questions. As a progressive technology in cognitive science, VR can help scientists study SM and paw this way. In this study, we have addressed one of the main constraints of creating VR environments for studies that involve body motion. Creating body motion in virtual environments can be challenging due to the technical limitations of motion tracking. We used IK to solve this problem. Although the IK method is readily used for gaming purposes, it is not an ideal solution for generating precise body movements. Using IK with a limited number of sensors can cause deviation from natural movements. To use this simplified method, we need to evaluate its efficacy in eliciting a sense of synchrony.
We have created a VR environment where a VC could mimic the participants after different delay times according to their study group. We used an IK package in the Unity Game engine to produce VC movements. To evaluate the effects of synchrony on participants, we asked rating questions concerning reported synchrony and social closeness. Our results showed that people in SG perceived more synchrony than in UG. The analysis of participants’ movements showed no difference in overall physical activity; thus, the discrepancy in reported synchrony between our groups is not due to it. Although we have not demonstrated any significant difference between the two groups regarding social closeness, there was a robust correlation between social closeness and self-assessed degree of synchrony within each group.
Our findings show that by manipulating the delay time of mimicking VC and using the IK method to animate it, we can successfully change the feeling of synchrony towards VC. The strong correlation between the self-assessed degree of synchrony and social closeness was not surprising, as previous studies have indicated the prosocial effect of synchrony. However, the absence of difference in social closeness between our groups, which may sound contradictory to previous findings, can be explained by the difference in properties of our method. Firstly, we used periodic movements that lasted more than the delay period; thus, after the delay passed, VC started to mimic, and at some points, the movements could get synchronized by the participants. Secondly, we used musical rhythm and asked participants to make their moves according to them. It can induce more sense of synchrony compared to a situation with no rhythm. In other words, the rhythm could provide additional sources for getting more synchrony in the VR environment. Thirdly, the impact of music, which has prosocial effects, cannot be distinguished in this experiment since participants did all movements by the same rhythm in each group. From previous studies, we know music can alter people’s feelings toward their conspecifics (
Stupacher et al., 2017).
We showed that without using sophisticated and expensive techniques of motion tracking, experiments involving interactive body motion in VR are feasible. Our method for simulating body motion on VC can be done using only three sensors available on almost any VR device. We have minimized the aesthetic details in producing our method, making creating such environments more straightforward for researchers. Other related questions to synchrony can be met by our method (for example, emotional facial expressions can be used to evaluate the effect of synchrony on affective processing, or different ways of manipulating synchrony can be tested). In our method, we have not limited participants’ movements. They could move freely and choose the order of their moves this way, we can create a sense of synchrony more implicitly. However, it led some people to find other external cues for experiencing synchrony despite our manipulation. The participants in the UG also reported a considerable degree of synchrony. This can raise interesting questions about the personality factors that cause people to feel synchronous toward others. Future studies can investigate the relationship between personal differences and self-assessed degree of synchrony in our paradigm since subjects are partially free to reach the desired level of synchrony by the VC.
There are several mechanisms suggested for the prosocial effect of synchrony. Some studies suggest synchrony can enhance the feeling of character similarity due to perceptual processes, which can improve prosociality. Other studies emphasize the impact of devoting more cognitive attention toward synchronous characters. The third mechanism suggests enhancing the effect of synchrony on positive emotions, which can be associated with the characters. Some studies also suggest the potential impact of synchrony on blurring the boundary between self and others. However, some evidence fails to show the mechanisms mentioned above (
Hu et al., 2022). Accordingly, we need to study the underlying mechanisms of synchrony in future studies to reach conclusive answers. A growing number of studies also focused on the biological aspects of synchrony, including pharmacological manipulation, physiological measurements, and neuroimaging. Our method can be used to study all of these open areas. For example, future studies can use VR and functional near-infrared spectroscopy to answer questions about the mechanisms of the SM since this imaging technique has good tolerance to motion artifacts (
Pinti et al., 2020). Finally, beyond research purposes, SM can have potential translational benefits. A growing body of research has been developed around exploiting the advantages of SM (such as dancing) to improve psychiatric disorders (
Meekums et al., 2015). Some studies have confirmed the positive effect on the quality of life in people with Alzheimer (
Ruiz-Muelle & López-Rodríguez, 2019).
VR is used to produce more realistic and immersive environments. From this perspective, using the IK method with few targets can be unfavorable since it is not entirely accurate. However, this issue can be addressed by using more sensors on the other joints. Future devices may provide additional sensors for the legs and trunk. Wearable accessory sensors can be used for body tracking in HTC Vive to improve movement tracking. When the experiment’s aim is focused on the pure effects of body movement synchrony, our paradigm is not robust. To solve this problem, we suggest using other types of creating asynchrony or limiting the movement periods to avoid overlapping them after the delay time passes. Using rhythm also made this paradigm weak in differentiating the prosocial effect of body synchrony between VC and subject from the synchrony between subject and rhythm. The impact of music also cannot be extractable here. Future studies may use other methods like vocal commands or visual signals to start and end the moves.
5. Conclusion
Increasing the self-assessed degree of synchrony with VCs can have prosocial effects. Participants who feel greater synchrony towards their virtual teammates rate themselves closer to it. According to the results, this effect can be created by VR environments without sophisticated equipment. The participants in synchronous groups perceived more synchrony in their movements toward their teammates.
Ethical Considerations
Compliance with ethical guidelines
This study was approved by the Ethics Committee of the Tehran University of Medical Science (Code: IR.TUMS.TIPS.REC.1402.047).
Funding
This research did not receive any grant from funding agencies in the public, commercial, or non-profit sectors.
Authors' contributions
Conceptualization: Milad Yekani; Methodology: Milad Yekani and Mehdi Tehrani-Doost; Visualization and software: Milad Yekani and Milad Rahimi; Resources: Mehdi Tehrani-Doost; Formal analysis, data curation, review and editing: Milad Yekani and Abdol-Hossein Vahabie; Writing the original draft: Milad Yekani; Validation and project administration: Abdol-Hossein Vahabie and Mehdi Tehrani-Doost; Supervision: Abdol-Hossein Vahabie.
Conflict of interest
The authors declared no conflict of interest.
References