Implementing Grab and Rotate Interactions for Models in Unity XR

This module focuses on a fundamental aspect of Extended Reality (XR) development: enabling users to interact with 3D models by grabbing and rotating them within a virtual environment. This is crucial for creating immersive and intuitive user experiences in Augmented Reality (AR) and Virtual Reality (VR) applications.

Understanding XR Interaction Basics

In XR, user interaction typically involves input from controllers, hand tracking, or gaze. For manipulating objects like 3D models, common interactions include direct manipulation (grabbing) and rotational adjustments. These actions need to feel natural and responsive to maintain immersion.

Grab interactions allow users to pick up and move virtual objects.

Users can 'grab' a virtual object by pointing at it with their controller or hand and pressing a button or performing a gesture. Once grabbed, the object follows the user's input device.

The implementation of grab interactions often involves raycasting from the user's input device (e.g., controller, hand) to detect when it intersects with a target object. Upon detection and activation of a grab input (like a trigger press), a connection is established between the input device's transform and the grabbed object's transform. This connection can be managed using Unity's physics system (e.g., FixedJoint) or by directly parenting the object to the input device's transform, ensuring smooth movement.

Rotation allows users to orient grabbed objects.

Once an object is grabbed, users can rotate it by moving their input device in a circular motion or by using specific controller inputs (e.g., thumbstick).

Rotation can be implemented by detecting the relative movement of the input device while an object is grabbed. For instance, if the user moves their controller in a circular path around its initial grab point, the grabbed object should rotate accordingly. This can be achieved by calculating the difference in rotation between the current input device orientation and its orientation at the moment of grabbing, and then applying that delta rotation to the grabbed object. Alternatively, using the thumbstick to control rotation around a specific axis is a common VR interaction pattern.

Unity XR Interaction Toolkit

Unity's XR Interaction Toolkit (XRI) provides a robust framework for implementing common XR interactions, including grabbing and rotating. It abstracts away much of the low-level complexity, allowing developers to focus on the user experience.

The XR Interaction Toolkit uses a system of components to manage interactions. Key components include:

XR Grab Interactable: This component is attached to any GameObject that should be grabbable. It defines the properties of the grab interaction, such as attach points and movement types.
XR Direct Interactor: This component is attached to the user's input device (e.g., controller). It handles the detection of interactable objects and the initiation of interactions like grabbing.
XR Ray Interactor: Similar to the Direct Interactor, but uses a raycast for interaction, often used for pointing at distant objects.
XR Controller: Represents the physical XR controller and provides input data.

When a Direct Interactor (or Ray Interactor) comes into contact with a Grab Interactable, and the appropriate input is detected, the Grab Interactable is 'grabbed' by the Interactor. The movement type (e.g., 'instantaneous', 'velocity tracking', 'kinematic') on the Grab Interactable determines how the object follows the controller.

📚

Text-based content

Library pages focus on text content

Implementing Grab and Rotate

To implement grab and rotate for a model in Unity using the XR Interaction Toolkit:

What Unity component do you attach to a grabbable object?

XR Grab Interactable

Add XR Grab Interactable: Select your 3D model GameObject in the scene and add the
code
```
XR Grab Interactable
```
component. Configure its properties, such as setting the
code
```
Movement Type
```
to
code
```
Velocity Tracking
```
for smooth physics-based movement. You can also define
code
```
Attach Transform
```
points for more precise grabbing.
Add Interactor: Ensure your XR Rig (the GameObject representing the user in VR/AR) has an
code
```
XR Direct Interactor
```
or
code
```
XR Ray Interactor
```
component attached to its controller GameObjects. These interactors will detect and initiate the grab.
Configure Input: Set up the input actions in Unity's Input System to map controller buttons (e.g., grip button) to the 'Select' action for the XR Interactor. This action triggers the grab.
Rotation: The
code
```
XR Grab Interactable
```
component, when configured with appropriate movement types and when the user's input device rotates, will inherently handle the rotation of the grabbed object. For more custom rotation logic, you might need to script additional behavior, often by listening to input events and applying rotations to the grabbed object's transform.

For intuitive rotation, consider using the 'Rotate Transform' feature within the XR Grab Interactable or implementing custom rotation logic based on controller thumbstick input.

Advanced Considerations

Beyond basic grab and rotate, consider:

Attach Points: Defining specific points on the object where the controller attaches for more natural grabbing.
Movement Types: Experimenting with 'Instantaneous', 'Velocity Tracking', and 'Kinematic' movement types to achieve different physical behaviors.
Haptic Feedback: Providing tactile feedback to the user when grabbing or releasing objects.
Multi-Object Grabbing: Implementing scenarios where users can grab multiple objects simultaneously.

Learning Resources

Unity XR Interaction Toolkit Documentation(documentation)

The official Unity documentation providing a comprehensive overview of the XR Interaction Toolkit and its core concepts.

XR Grab Interactable - Unity Learn(tutorial)

A Unity Learn pathway covering the basics of the XR Interaction Toolkit, including how to set up grab interactions.

Unity XR Interaction Toolkit: Grab and Release(video)

A video tutorial demonstrating the implementation of grab and release mechanics using the XR Interaction Toolkit in Unity.

Unity XR Interaction Toolkit: Rotation(video)

A video tutorial specifically focusing on how to implement rotation for grabbed objects using Unity's XR Interaction Toolkit.

Unity XR Interaction Toolkit: Input System Setup(documentation)

Documentation on Unity's new Input System, essential for configuring controller inputs for XR interactions.

Understanding XR Interaction Concepts(documentation)

Oculus developer documentation on interaction concepts, which are broadly applicable to VR development.

Best Practices for VR Interaction Design(blog)

Articles and insights on designing effective and intuitive interactions for virtual reality experiences.

Unity XR Samples Repository(documentation)

Official Unity GitHub repository containing project templates and samples for XR development, often including interaction examples.

Introduction to 3D Physics in Unity(tutorial)

A foundational course on Unity's physics engine, crucial for understanding how grab interactions with `Velocity Tracking` work.

Unity XR Interaction Toolkit: Advanced Grab(video)

A more advanced tutorial exploring custom grab behaviors and configurations within the XR Interaction Toolkit.