{"id":395,"date":"2014-05-06T02:02:09","date_gmt":"2014-05-06T02:02:09","guid":{"rendered":"https:\/\/projects-old.etc.cmu.edu\/bowtie\/?page_id=395"},"modified":"2014-05-12T22:27:58","modified_gmt":"2014-05-12T22:27:58","slug":"the-show","status":"publish","type":"page","link":"https:\/\/projects-old.etc.cmu.edu\/bowtie\/?page_id=395","title":{"rendered":"The Show"},"content":{"rendered":"<p><iframe loading=\"lazy\" width=\"792\" height=\"446\" src=\"https:\/\/www.youtube.com\/embed\/2ZPx2u7srlc?feature=oembed\" frameborder=\"0\" allowfullscreen><\/iframe><\/p>\n<p><strong>Reviews<\/strong><\/p>\n<p>The weakest parts were HERB not using fingers (like Yume Project at ETC seeing<br \/>\nthat HANDS were very important and resorting to gloves to improve robot<br \/>\nperformance), HERB not moving during actress lines for the most part (explained in<br \/>\nQ\/A), and lack of any quick reactions.<\/p>\n<p>The best parts for HERB were its backing up and reacting to the long<br \/>\naggressive delivery of the actress lines (hints that a play where HERB would have<br \/>\nhad time to react to others&#8217; lines would have been better choice), and head pose<br \/>\nmovement to make more contact with stage action. I liked the Hemingway limb<br \/>\ngestures and the big &#8220;I like to party&#8221; limb gesture. Big and expressive worked, the<br \/>\nfirst time used. If repeated too often, it would give away that HERB does not have<br \/>\na huge set of combinatorial expressions like a human actor would.<\/p>\n<p>&#8211;Mike Christel<\/p>\n<p>&nbsp;<\/p>\n<p>One of the things that kind of struck me was timing, obviously there&#8217;s frustration with the robot, One the question I had was differentiating between using the technology of the robot to move the arm and take advantage of that, versus using the robot to create a good well-timed back-and-forth repertoire with the actress, and to basically provide technological transparency, so the audience wouldn\u2019t be thinking the robot is not doing what it should be doing, but the audience will be thinking the robot is responding, and responding so fast that I\u2019m now sure if it\u2019s automated? Or someone is real-time puppeteering it, or it\u2019s pre-scripted, you are not thinking about the technology you are thinking about response of the robot in the context of the performance.<\/p>\n<p>&#8211;Sabrina Haskell<\/p>\n<p>&nbsp;<\/p>\n<p>One of the perspectives from the performance side, I think you guys really did well on the technical side of stuff. The tool, the technology and how you get to work was the core, but the performance was the a thing that you have to do, but wasn\u2019t the core.<\/p>\n<p>&#8211;Dave Culyba<\/p>\n<p>&nbsp;<\/p>\n<p>We are relieved a little bit but also really happy the way we finished this semester. I think the lab was . Jon and our team,\u00a0we are thinking what even outside the performance.<\/p>\n<p>&#8211;Mac<\/p>\n<p>&nbsp;<\/p>\n<p>Simon started animating using the blender plugin, he animated HERB in each case. They also realized which part of HERB is important, like fingers, in conveying emotions I think that\u2019s the biggest take-away.<\/p>\n<p>&#8211;Jon<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p><strong>HERB Performance Q&amp;A Documentation<\/strong><\/p>\n<p>Transcript by Robin Li<\/p>\n<p>Q:(Sam) Before we end the show tonight, I think one the most exciting thing about the representation is the two diverse audience we have tonight, so I want to open up for questions, that can be about the theater side, the robotic side, and sort of see where the conversation will be going. I\u2019m actually gonna start it though, I\u2019m gonna be the first button, so Don, who has actually acted in high school, does this tonight feel like acting?<br \/>\nA:Yeah definitely. It was of course it\u2019s different in that when you are on stage you have to make choices about what to do and when to do them at the same time, but here with the robot I already made choices about what to do since I already animated them beforehand. The choice that I have at the moment was when to do them. Of course sometimes HERB wasn\u2019t quite cooperating.(audience laughter\u2026) but after some time I kind of got more excited\u2026<\/p>\n<p>Q: You said HERB sometimes wasn\u2019t cooperating, could you talk about what the pauses are?<br \/>\nA:(Don) When it pauses, it usually means that I started a queue before the last one finishes. It hangs. (Sam) Yeah, HERB has to return to the idle spot for most of the next ones. The bigger ones take longer to the idle spot, Don is finding the balance between waiting too long for a queue to settle<\/p>\n<p>Q: Is your software supporting listening and speaking, is there a way HERB could listen to Olivia and hear when she stops speaking so the need for an operator will become less immediate after each queue?<br \/>\nA: (Garth) I\u2019d love to do that, but it\u2019s mostly a concern about reliability and robustness in the theater. Here theater is a much more demanding demo than robotic demos. The thought of having it work every time the best we can motivates us to keep a human in the loop. Also human is in the loop to have the high level interaction, Sam says no robot could ever understand that, that\u2019s human understanding. So yes, it will be nice to work on speech recognition to be much more robust enough to at least answer a dialog at a time. The speech engine we build is actually capable of variation on the performance, the process of building a synthesized voice also involves analyzing the speech in a way that we could reconstruct it. We also like to extend more of our mean stretching the interpretation of the dialogue This was because basically the original interpretation from the actor in the sound booth was so good that we didn\u2019t manipulate it very much, that was sort of the counterpart to this\u2026<\/p>\n<p>Q: Did you consider allowing HERB queued by the other actor? So essentially being able to reconstruct a one person show ?<br \/>\nA:It\u2019s completely technically feasible, Don is like clicking one button\u2026<\/p>\n<p>Q: So this performance was not autonomous, sort of looking forward, would it be possible to have a robot improvising?<br \/>\nA:(Garth)You get these questions of human intelligence very quickly, there are these contents every year people write chat box or other programmers trying to interact and dialogue with people, basically simulating being a human, every year they get better, so yeah, technology is there with voice recognition and kind of dialogue recognition too, to build believable agent that could do the same, that\u2019s an entire research path you can delve. We are as a lab kind of focusing more on the physical expression, the arm gesture, for me the interesting part here is taking the non-anthropomorphic or at least semi-anthropomorphic robot, then find movements in it without emphasizing the face and more emphasizing the arm and body language to create a visual vocabulary, instead of pushing that to the dialogue and story telling aspect, but that\u2019s also a part of it.<br \/>\n(Sam) To elaborate from director\u2019s perspective, I thought it\u2019s more interesting to develop a robotic puppet than a robotic actor, who knows maybe in 30 or 40 years,\u2026(can\u2019t hear clearly here)<\/p>\n<p>Q:What was the original learning objective of the project? (silence for 2 seconds, then burst into laughter, first Garth then everybody)<br \/>\nA: In some sense this project is the sort of fun project that evolves to see really where it could go. But at the same time it is serious, you got the questionnaires, there are obviously some intentions behind there. What Personal Robotic Lab is doing is trying to figure out how robot behaves in everyday life situation, the interaction with people is such a part of that, a condensation of drama in a specific form, out of that, we discovered something about how to tell a story, how to convince some human around about what you are to do, obviously it has nothing to do with normal daily activities, I would like to see now is HERB grab things at the table, not just does it in the way that accomplishes the task, but in a certain way that expresses itself as well, now we have the library of movements and ideas of how HERB can move. I also want to point out Sid is the PI of the lab, much of this is his agenda. More questions?<\/p>\n<p>Q:(unclear but is about HERB\u2019s joints in front, and how it affects the acting)<br \/>\nA:(Sam)That was sort of a learning experience in itself. The original thing that we tried was to simulate human movements as close as possible, like try to make HERB shrug like HERB can shrug as if he has shoulders(laughter). Then we started to see what is HERB\u2019s equivalence of a shrug. Sort of what is a \u201cnon committal move that doesn\u2019t require a lot of energy\u201d figuring out how to move his elbows in the way that says \u201cNay\u201d(Sam acting shrug)<br \/>\n(Garth) HERB is really designed for functional purposes, two arms can do bimanual functions, like moving objects with two hands. The head is actually added recently, responsible for practical tasks. But you know we can find emotions in it, that\u2019s sort of the lesson with that, any interesting mechanism you explore it as a puppet and find the expressive emotions involved.<\/p>\n<p>Q: I\u2019m just curious in the animations HERB\u2019s fingers never move? He has fingers<br \/>\nA: (Jon) I think the fingers were moving in the simulation.<br \/>\n(Aaron) HERB has three fingers, it can be kind of large, again he is designed for functional purposes, in some cases it was kind of tricky to open them up and do these gestures without looking like claws\u2026<br \/>\n(Sam) He can demonstrate that\u2026<br \/>\n(Don demonstrated a open fingers animation, audience burst into laughter\u2026)<\/p>\n<p>Q: I noticed that HERB didn\u2019t have huge react of dialogues, the question for you is was that a technical issue or was that done by choice? Then I want to ask Olivia that question as well.<br \/>\nA:(Garth) You mean he isn\u2019t directly responding to\u2026(Q) While she is talking, he is doing nothing.<br \/>\n(Sam) There were actually a few times some smaller ones, it was during the monologues, it\u2019s mainly a technical thing, most of time he was using the length of shorter lines to get back into the initiation for the next one. For the longer lines, he had a few looking arounds, looking downs, but for less than 2-3 seconds long will be pushing it get all those things.<\/p>\n<p>Q: As an actress, how was it to perform with an actor who wasn\u2019t reacting to you?<br \/>\nA:(Olivia)En&#8230; I think it\u2019s a really interesting question, because it actually never occurred to me as an actress that he wasn\u2019t, there\u2019s obviously a point to which, this is not, as you said, a semi-anthropomorphic robot, there\u2019s a obvious element of non-humanness, but certainly as an actress you have to accept what you are given, we talk about the reality that this is in the caffe which isn\u2019t, in our reality it\u2019s totally weird this robot is just coming out and talking to me. Or\u2026 some time in the future, it\u2019s totally average that robot comes talking to me, or some other dimensions, some other reality where as before this is our world, but this is just another character in that world. I think that\u2019s the ones that we landed on as such, while it would be really interesting to explore what human reactions would be to robot in situations like this\u2026<\/p>\n<p>Q: Kind of a question looking into the future, what kind of advances would you put on makes it easier to do today than back then?<br \/>\nA: I mean, honestly what it comes to down to, is the open source movement, because much of the software we have built has been based upon a wide variety of open source software, we leveraged blender the 3D animation package, using ROS the robotic operating system, at some sense all of the tools we are building are based on the works of thousands of people, arguably that\u2019s more cultural , in terms of technology, we probably could run this on 10 year old computer in some respects, it wouldn\u2019t be that different, the very arm on HERB was actually designed in the late 80\u2019s. The technology has been there for long time. So I don\u2019t think we are really using specific high right now, we are able to work in the eco system of robotic technology which is evolving, that\u2019s the real difference.<\/p>\n<p>Q: Well first of all, on behalf of school of drama, want to thank everyone who\u2019s not in the school of drama to bring this in, and sharing with us and I think you guys have done a great service. My question is out of the context, I\u2019m the guessing the primary purpose of the robot is helping disabled people in their homes, I\u2019m wondering if you guys have done field tests to actually bring this to people at their homes, what their reactions have been and wether they had trouble accepting it?<br \/>\nA:(Garth) This is actually a question for Sid.<br \/>\n(Sid) Thank you for the question, it\u2019s a great question, we\u2019ve done several of these studies, the evolution of the robot itself has been motivated by a large number of these studies, to get the robot to work with people, not just scientists like us, but real people(audience laugh), But\u2026the result has been startling sometimes, some people have found the robot endearing, some people have found the robot threatening. We are playtesting with children, actually children in the science center, 3 or 4 year old kids, who come by and some are immediately fond of the robot, and some\u2026there\u2019s a great anecdote, people were playing with the robot, the little kids, and then he started speaking, he said \u201cHi, I\u2019m HERB\u201d, these little kids started hugging each other (audience laughs), I\u2019ve never seen such thing like that, the reactions have been very interesting, now we want to use the tool of the dramatic performance as a way to understand more about how the robot is not just being functional, but also be expressive and emotive in what he does.<\/p>\n<p>Q: So what\u2019s next in terms of making it more engaging for people?<br \/>\nA:We are building the scaffolding for dramatic performance, we are developing open source tools that can allow anybody, not just us, but even any kid with blender to create animations and mount them on the system, tremendous amount of tools we build we want them to be open source, and again like Garth was saying, we want to take the learning from this and automate the process, because all of you gave very excellent suggestions on what we should automate, we\u2019ve been thinking about these, I want to take these and also put it into HERB\u2019s everyday activities, just like if he\u2019s picking up a coffee mug, if he picking up the coffee mug happily or sadly, or nervously, I think that\u2019s what it really takes to get robot like this not just doing things in people\u2019s homes but being accepted in people\u2019s homes as an agent they want to work with. (Garth) One last question\u2026how about you being the last one right here\u2026<\/p>\n<p>Q: You said you want to make all the softwares open source, how specific is the current software for HERB, or can it be adapted basically such that you can say this point of blender animation equally works on any other robot?<br \/>\nA: (Garth) Of course there are ad-hoc nature of the show, there are some kind of things that are still HERB-specific, at the core it\u2019s pretty general, it\u2019s based on ROS which is actually designed for a variety of robots, description of an arbitrary robot, we haven\u2019t done it yet so we can\u2019t promise, the goal is to have that, so with that I think we could close the formal session so people could go, oh sorry\u2026 (Olivia and HERB took a bow, audience applause\u2026) please fill out the final part of your survey for us, come down ask questions if you like.<\/p>\n<p>Transcript by Robin Li<br \/>\nThere might be minor errors due to recording quality, corrections are welcome<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Reviews The weakest parts were HERB not using fingers (like Yume Project at ETC seeing that HANDS were very important and resorting to gloves to improve robot performance), HERB not moving during actress lines for the most part (explained in Q\/A), and lack of any quick reactions. The best parts for HERB were its backing <a class=\"read-more\" href=\"https:\/\/projects-old.etc.cmu.edu\/bowtie\/?page_id=395\">[&hellip;]<\/a><\/p>\n","protected":false},"author":4,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"class_list":["post-395","page","type-page","status-publish","hentry"],"_links":{"self":[{"href":"https:\/\/projects-old.etc.cmu.edu\/bowtie\/index.php?rest_route=\/wp\/v2\/pages\/395"}],"collection":[{"href":"https:\/\/projects-old.etc.cmu.edu\/bowtie\/index.php?rest_route=\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/projects-old.etc.cmu.edu\/bowtie\/index.php?rest_route=\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/projects-old.etc.cmu.edu\/bowtie\/index.php?rest_route=\/wp\/v2\/users\/4"}],"replies":[{"embeddable":true,"href":"https:\/\/projects-old.etc.cmu.edu\/bowtie\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=395"}],"version-history":[{"count":5,"href":"https:\/\/projects-old.etc.cmu.edu\/bowtie\/index.php?rest_route=\/wp\/v2\/pages\/395\/revisions"}],"predecessor-version":[{"id":422,"href":"https:\/\/projects-old.etc.cmu.edu\/bowtie\/index.php?rest_route=\/wp\/v2\/pages\/395\/revisions\/422"}],"wp:attachment":[{"href":"https:\/\/projects-old.etc.cmu.edu\/bowtie\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=395"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}