{"id":295,"date":"2018-11-24T22:04:43","date_gmt":"2018-11-24T22:04:43","guid":{"rendered":"https:\/\/projects-old.etc.cmu.edu\/pupil\/?page_id=295"},"modified":"2018-12-11T22:42:55","modified_gmt":"2018-12-11T22:42:55","slug":"3d-line-prototype","status":"publish","type":"page","link":"https:\/\/projects-old.etc.cmu.edu\/pupil\/index.php\/3d-line-prototype\/","title":{"rendered":"Prototype 1b – 3D Line"},"content":{"rendered":"\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t

Prototype 1b - 3D Line Demo<\/h1>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
<\/div>\n\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t

Objective<\/h2>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t

To explore a gesture language for learner-content interactions in the domain of coordinate geometry.<\/span><\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
<\/div>\n\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t

Platform<\/h2>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t

Oculus Rift + Zed mini + Leap Motion<\/span><\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\t\t\t\"\"\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
<\/div>\n\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t

Development Process<\/h2>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t

To explore a gesture language for learner-content interactions, we decided to create a visualization of a 3D coordinate system that the students could directly interact with. We started out considering a number of interactions, which we had to scope for our time frame.<\/span><\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\t\t\t\"\"\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t

Implemented Features<\/h3>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\"\"\t\t\t\t\t\t\t\t\t\t\t
Move the Graph and Line Equation<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t

As the user moves their hand towards the origin, a reactive handle appears that can be pinched to move the graph around. This was useful since we weren\u2019t able to track images to spawn the graph at a specific location, and the user might want to move the objects to a more comfortable location.<\/span><\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\"\"\t\t\t\t\t\t\t\t\t\t\t
View Coordinates<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t

The user can point within the bounds of the coordinate space to display a tooltip that indicates the X, Y and Z coordinates of the point tracked by their fingertip.<\/span><\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\"\"\t\t\t\t\t\t\t\t\t\t\t
Modify The Line<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t

The user can also modify the line, either dragging it with a single hand or changing its orientation using two hands. The equation reflects the change.<\/span><\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\"\"\t\t\t\t\t\t\t\t\t\t\t
Move Bounds of Coordinate Space<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t

The user also has the ability to extend the bounds of the coordinate space by moving the arrow handles at the ends of each axis.<\/span><\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
<\/div>\n\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t

Results<\/h2>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t

Responding To The User<\/h3>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t

We learned a number of lessons on how best to respond to the user for ease of use. <\/span><\/p>

An early win for us was the pinch handle at the origin, which allows the user to reposition the whole graph appears as the user\u2019s hand approaches. It clearly show its affordance through its shape, and it prompts the user to pinch it by moving towards the user\u2019s hands. It also provides some leniency from tracking inaccuracies that may occur on the Leap Motion.<\/span><\/p>

A line shader for the 3D line that responds to the user\u2019s palms was also important in reducing overall frustration. A flat shaded line is hard to locate in space, and may exhaust a user that is unable to correctly perceive it\u2019s depth, when it gives no indication on how close or far the user was from the required position.<\/span><\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\"\"\t\t\t\t\t\t\t\t\t\t\t
The line with and without the shader<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t

When we added the shader that responds to user\u2019s hands, it allows the user to subconsciously estimate hand placement, making the experience a bit more ergonomically comfortable. In addition, when the user successfully pinches a point, a small \u201cexplosion\u201d effect appears, which confirms the interaction. Similarly, using the dotted lines as guides for when the user is observing a point in space helps the user locate themselves in the space better.<\/span><\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\"\"\t\t\t\t\t\t\t\t\t\t\t
Early version of \u201cexplosion\u201d on pinch<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t

Separation of Inspection and Manipulation of Virtual Objects<\/h3>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t

An important distinction when dealing with the 3D line model was the need to be able to \u201cinspect\u201d elements, like viewing the coordinates of a point in space, along with the need to \u201cmanipulate\u201d elements, like changing the orientation of the line. To achieve a separation between these two interactions, we used a pointing gesture to inspect, and reserved the pinching or grabbing gestures to manipulate virtual elements. This works out especially well so that users can feel comfortable inspecting specific points on the line without being worried about accidentally changing its slope and having to reorient the line.<\/span><\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t

Visual Affordances of Interactables<\/h3>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t

Clearly Visible Interactables<\/h4>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t

It is important to have points of interaction that are clearly identifiable and visible. As a gesture language for AR becomes more common, it may be easier to have smaller UI that does not distract from the content of the experience. However, the point handles to move the equation of the line were hard for any users to spot. A possible solution could be to make the points a brighter colour or larger in shape so that they stand out more.<\/span><\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t

Size and Shape<\/h3>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t

The axis handles were too big for their intended interaction, and suggested to users that it should be grabbed with an open hand. <\/span>\u00a0<\/b>However, the resulting constrained movement (of the cone locked to axis line) resulted in a lack of control and feeling of glitchiness. On top of this, there was no visual feedback of whether the handle was grabbed or not. \u00a0It is possible that if the handles were shaped differently or were smaller, users would assume a more accurate interaction (pinching), and interact with more of a purpose in mind.<\/span><\/p>

Another learning here is that if a user grabs an object with their full hand, they probably assume they will have unconstrained control of it. Smaller objects, or objects that are attached to things like levers or buttons, usually convey their physical constraint more clearly, ensuring that there is less of a disconnect when users interact with them.<\/p>

A good future work item might be thinking about how to visually convey that the motion of an object is constrained to a line – perhaps using a ring instead of a conical arrow?<\/span><\/p>

In all, sometimes the common \u201cassumed\u201d representation of a concept can collide with the visual affordances of its interface. In this case while the cone object fit well with the notion of arrows at the ends of an axis, it did not map well to its sliding affordances.<\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t

Positioning Objects in Physical Space<\/h3>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t

The positions of the virtual objects play a big role in the user experience. As this was our first prototype where we did not use any real world tracking to spawn the object, we allowed the user to move all virtual elements (the graph and the equation) and position them as they please. This, however, did sometimes lead to situations where the original spawn location was highly uncomfortable (e.g; passing through the user\u2019s body), which the user could then change.<\/span>
<\/span><\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t

Text in AR<\/h3>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t

While not exactly a new learning, getting text right in AR does require some iteratio. We opted for a simple drop shadow to separate our white text from light backgrounds, but perhaps legibility could be made better by always placing text on contrasting solid plates.<\/span>While not exactly a new learning, getting text right in AR does require some iteratio. We opted for a simple drop shadow to separate our white text from light backgrounds, but perhaps legibility could be made better by always placing text on contrasting solid plates.<\/span><\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t

Tracking Difficulties<\/h3>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t

The Leap Motion some limitations that affected usability in this context. High school students we tested with were frustrated with the pinching interaction as opposed to grabbing the line.<\/span><\/p>

It is also possible that the user\u2019s hands move out of the tracked area while performing an action like rotating a line, which causes some frustration. Indicating to the user that the camera can\u2019t see your hand, and therefore doesn\u2019t know what you\u2019re doing, was something we did manually for our scope, but is a problem that could be solved with some indications in the experience itself.<\/span><\/p>

The Leap Motion maintains the state of your hand at the last known state if it can no longer track your hand. This means that if the hand moves out of the tracking area while the user was pinching it, and the user then releases it, the Leap Motion does not register that change and continues \u201cgrabbing\u201d the object.<\/span><\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t

Future Scope<\/h2>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t

From our early ideas, there were a number of features we discussed but didn\u2019t have the chance to implement:<\/span><\/p>