Olga Sorkine-Hornung, Belen Masia and Yuting Ye share their journeys at the 2022 Berthouzoz Lunch panel

Yuting: I work at Meta as a research scientist. My team focuses on creating virtual humans for AR/VR/XR. My background is in character animation, and I am excited about the possibilities of what I can create and how I can make things move. After graduate school, I developed animation and rigging tools to support artists at Industrial Light & Magic (ILM). It was a great opportunity to observe how artists work and gain an appreciation for the practical problems they encounter that we as researchers typically do not. I moved to Facebook when they had just acquired Oculus. I was very excited to join the company since I think VR opens up a new dimension to create animation and it is still a new and exciting medium for computer graphics.

Olga: I have been a faculty member at ETH Zurich for 11 years. I was born in Russia and my family immigrated to Israel when I was 12 years old. I was always attracted to mathematics as a kid. My goal in life was to find a well-paying job as soon as possible, so I picked computer science and math for my Bachelor’s studies at Tel Aviv University in order to become a software engineer ASAP. In my third year, I took Daniel Cohen-Or’s computer graphics course, and I was shocked that such simple math could create such beautiful images and produce such interesting things. I was hooked on graphics! After my Bachelor’s, I had to do mandatory military service, but I asked Daniel if I could take his classes in the evenings, so I started my Master’s studies during my military service. When I was released two years later, I signed up for the Direct Doctorate program in order to get funding to take more classes. I got hooked on research, and although I thought I wasn’t a very strong programmer, I wanted to continue pursuing academic research. I did my postdoc in Berlin with Marc Alexa, then I got my first job as an assistant professor at NYU, then I moved to ETH Zurich in 2010. I work on geometry processing, geometric modeling and deformations, and related topics like animation. My group is IGL, the Interactive Geometry Lab.

Belen: I am a professor at the University of Zaragoza in the Graphics & Imaging Lab. My undergrad degree is in engineering. I loved problem solving and math and physics, so engineering was a logical choice, but during undergrad I discovered programming and fell in love with it! When I entered undergrad, I thought I might become a project manager, but then I discovered research as an option and loved the fact that researchers could actually choose the problems they wanted to face and find solutions to them. During my studies I was lucky enough to spend time at places like MIT and Max Planck, but I always knew that I would return to Spain. The moment that a tenure-track position in Spain appeared, I went back. I have never been in industry because academia is such a fantastic place to be – we may get paid less, but we have so much freedom to choose our topics and collaborators.

Belen: There are many paths to an idea. Indeed, sometimes the inspiration comes from art or nature. For example, in 2013 we worked on a project that was led by MIT, investigating femto-photography. Femto-photography is an imaging system that is capable of capturing a trillion frames per second – fast enough to capture light in motion propagating through a scene. A source of inspiration for that project was the work of Harold Edgerton, who shot famous photographs of bullets going through apples. There was a parallelism there with femto-photography, which captures light in motion, the way Edgerton captured objects in motion.

Those are not necessarily the usual cases though. Ideas usually come from a lot of thinking and reading what other people have done. You have to understand the problem, question every step along the way, think a lot about it, and eventually the ideas will form. There is usually an “aha!” moment. But this usually comes after a long, iterative process and a lot of thinking.

That said, there are millions of high-level ideas and it’s easy to have them… as students know well, rolling your sleeves up and getting the work done is the toughest part!

Olga: Many people are inspired by certain applications or art, but I have to admit that deep down my inspiration is the curiosity of mathematics. Often I need to invent a justification to explore a mathematical idea as an afterthought or to apply for a grant. For example, the idea for my work on Laplacian mesh editing (a deformation method that uses simple linear algebra to deform meshes) came from a previous project about mesh compression – we would extract the differential coordinates from a mesh by applying a Laplacian at every vertex and quantizing those coordinates, and in order to reconstruct the geometry from the quantized coordinates you have to solve a constrained linear system. As I was working on that paper, I thought, what if I don’t quantize anything and just move those anchor vertices to a different place and solve the linear system? Well, the mesh gets deformed. I was bored, I fiddled with the code to see what would happen, and then I explained what happened with the mathematics. That’s how the Laplacian mesh editing project was born.

For the past few years, I have been really interested in the application of virtual or real garments and fabrication of textiles. I am a bit obsessed with clothes and fashion in my private life, and then I realized that I am tenured now and I can do whatever I want (laughs). Why don’t I put my whole being into my job? So I am working on modeling custom-made clothing and fabrication problems in textiles.

Yuting: For me, it’s an evolution of one thing leading to another – it always turns out there is more to be done! When I was in grad school, I studied physics-based character animation, which is a very simple physics principle of F = ma. But as soon as you add 3d rotation, or go from a box to a real person, things get quite complicated. During my PhD work, I often had to ask myself, “what is the minimum set of physical equations or principles that would allow me to generate realistic character motion?” This was largely because computing power was quite limited, so we could only do a complex optimization problem with a small number of degrees of freedom. Still, it turned out we could do quite a lot!

A PhD student recently asked me why I chose the projects I did in grad school. I told him to try to choose just one simple principle and apply it to different things. He told me that it is not obvious at all from the papers that one principle manifests in so many different ways.

After my PhD, I realized that physical principles describe only one aspect of motion; the other dimension is style, because everyone walks differently. An artist looks at motion completely differently from how we look at motion from a physics perspective. Then the question becomes: do they have the degrees of freedom or leverage to specify what they want given the physical constraints? And that problem itself is very complicated. I worked closely with animators and riggers at ILM to see how they work, but the tools they use are unintuitive and far removed from the final product. They manipulate sliders and knobs and build a mental model of how those translate into 3d motion, but that is a really painful process – I wanted to change how they work and make their tools more intuitive. But I was limited by the input technology of the time.

When I moved to Facebook, I worked on a hand-tracking team that could track hands and fingers in 3d so that users could have a very intuitive interface to manipulate motion. I spent five years shipping this hand-tracking feature on the LVR headset. But once you ship a product, it almost becomes a liability that you have to continuously maintain and it becomes more of an engineering or mechanical job.

To switch it up, I moved to a team that is working on full-body tracking to answer questions like: if we have the full degrees of freedom of a body in 3d, what kind of input or interface or creative process can we enable with this new medium? I believe that everyone can create and be inspired if we just provide creation tools that are easier and more intuitive to use. I also believe that having more creativity and imagination will make the world a better place!

Olga: Grad students are conditioned to focus on the next SIGGRAPH paper and it is hard to see beyond the one-project horizon. But as a professor, I have the pleasure of writing funding proposals. These are good exercises to present a longer-term vision. In practice, I find that having a rigid, clear vision can be limiting – I am a dynamic person and I try to adapt the research to each member of the group and their strengths and preferences. So rather than having a clear, planned vision, we have general directions that interest me and my group, like developable surfaces or garment modeling or character animation. Then I let everyone find something that they connect with in that space. What helps me is having a guiding set of principles of how we work, and then we can find specific things to do and do them well. For example, I do not like to work alone and thrive in collaboration, so I try to create small teams for each project. For every project, there needs to be at least one exciting mathematical nugget. We also strive to work to the highest scientific standards, including going deep and admitting what we don’t understand. Given these general principles, the vision writes itself. It helps younger researchers to be given a clear direction, but the ultimate goal is to grow and become independent and thrive with our own inspiration. I try to be open and flexible.

Belen: I could not agree more with Olga, point by point. I will comment on the instant gratification vs. long-term – in that regard, if you only enjoy the destination and not the journey, it will be a very tough journey. The journey is the good part. Whether the destination is the SIGGRAPH paper or tenure, you have to find a way to enjoy the hardest parts: thinking and implementing. Of course, it can be frustrating at times, especially during a PhD. You have to be resilient, but it is also very important to have the right support, both from supervisors and your research group. We try to collaborate a lot so that the students feel supported by their colleagues.

It is fine to strive for a blue-sky, long-term vision, but sometimes this vision is conditioned on the fact that we still have to write grants and find funding. This is the world we live in. Although we would love to lose ourselves in trying to understand these very specific problems, at the end of the day, someone is paying for what you do, so you have to demonstrate that what you do can have an impact on society.

Yuting: I can talk a bit more about the industry side – I totally agree with Olga, it is not always helpful to have a rigid long-term vision. I just want to solve problems, but there are so many problems out there! I try to find some common themes among all the problems, then find a technical nugget in there in order to address several of those problems systematically. In the industry, we have to define very concrete milestones and each milestone needs to be convincing. The person you are pitching to may not have the technical background to understand what the solution is, but you can try to describe the small problems along the way and how each leads to the next, in order to convince them that your plan is feasible. Since we are solving practical problems, our plan also changes and evolves in response to the feedback we get from users along the way. We may realize that our original thinking is completely different from how it actually works, or perhaps our solution changes the way others think about the problem and then the follow-up problems become a different set of things. It is rare to have a grand vision and then solve it. More often, you are curious about a few things and start to work on them, then over time a theme begins to emerge because we are people and we are predictable and coherent, and that becomes your research agenda.

Yuting: It is very important to have a role model – personally, I have been trying to become my advisor! Many years after graduating, I still hear her voice in my head when I have a problem and her voice tells me what to do. I feel really fortunate to have had a wonderful advisor and I learned a lot from her: how to look at problems, how to do research, how to formulate ideas in a systematic and clear way, and more. You can learn bits and pieces from everyone, so try to interact with more people, have a learning mindset, and find something to take away from every interaction.

It’s also helpful to do internships in different environments because you naturally get to work with a different set of people and are exposed to new problems. This broadens your perspective of the problem space and puts it in a bigger context.

Olga: I was fortunate to have an advisor who is an idea machine. The problem was that I relied so much on his ideas and it was easy to know what to work on. My confidence to develop my own research ideas and projects came relatively late – what helped me a lot was stepping into the shoes of an advisor. When I was a postdoc, I advised Master’s students for the first time and although it felt unnatural to me, I had to pretend to be the responsible adult and that forced me to start working independently because someone was relying on me. Teaching and advising really helps. I think I only really started understanding linear algebra when I started teaching it at ETH; this is true for research as well.

I especially want to echo how important it is to read and communicate with others. I think independence comes from being well-read. My students will laugh at me because it is my constant source of frustration, but never rely on your advisor for the prior work section! We don’t necessarily have time to read all the literature. Reading a lot is very inspirational and helps to develop your own vision. Also, remember that there were papers written before 2017!

Belen: Your advisor will have a lot of ideas and will guide you, but try to build an opinion for yourself. Your advisor by no means holds the truth always! You will become the expert on the topic you are working on, and maybe eventually even more of an expert than them. Question them. Offer your opinion. You can even propose collaborations or pursue internships, which are great ways to be exposed to other environments. These force you to have an opinion on other topics.

Once you finish your PhD – and this was strange for me since I am now a professor in the same place where I did my PhD – you have to separate yourself from your advisor. Try to find lines or topics that are your own: you are not starting from scratch, you will probably start with what you did during your PhD, but you try to find your niche in the whole ecosystem.

Yuting: I am generally pretty motivated – there are just so many interesting problems out there! But if it does happen, talk to others. Let people around you inspire and motivate you – ask them, what are you frustrated with today, and how can I help you?

Olga: During my PhD, we didn’t have as many tools and libraries as there are now. I always try to reduce the problem to the smallest possible core and prototype that core as quickly as possible. I use MATLAB to try out the most basic functionality of my idea – no UI or bells and whistles – and once I convince myself that the idea is promising, then I invest the effort to write the surrounding software. Some of my labmates were also much more proficient in linking and libraries – in this case, team up! All of us have different talents and superpowers!

At the beginning of your PhD, I encourage you to invest time in learning tools that are relevant to your research direction – this can help you a lot later. I wish I could tell that to my younger self.

Belen: Not during my PhD, for sure! As a postdoc, you start to see what it is to be on your own. You have an advisor, but it is the first time you are expected to come up with your own problems and ideas. When you start supervising students and see them finish and do well, you feel like you’ve set a path for them and they’ve followed it (of course, they contributed a lot!) and you contribute to them becoming independent researchers.

Olga: Fake it ‘til you make it! Applying to faculty jobs was terrifying, and it was even more terrifying when I got faculty job offers. It started dawning on me that I was my own professor when I was at NYU – everyone was so welcoming, and Denis Zorin (my idol and a legend in geometry processing) was treating me like a peer. Then I started thinking, if he’s treating me like this, I better get my stuff together and start acting like it!

Of course, it is different for different people and depends on the advisor. In the European system, labs tend to be bigger and students get less supervision, so they have to grow up much faster. We on the panel had the privilege of very personal, hands-on advising – that might be why I was a late bloomer.

Yuting: It is your choice how much you want to work. As grad students, we get used to the lifestyle of always working – some people enjoy it, others feel guilty if they take time off to do something fun. But you have the power to make that decision. I have heard from colleagues that having kids helped them learn how to be more efficient and prioritize because that is a hard constraint – you have to meet that constraint and figure out in the null space what you want to do!

Belen: Honest answer? Badly. You are your own boss, for better or worse. We have a somewhat competitive environment and you have to juggle teaching, funding, and so on – it is a lot of multitasking and tasks are open-ended. You could always be doing more. I have tried to be very strict and disciplined and stop after a certain amount of hours, otherwise it can go on forever.

Olga: I have also heard that when you have kids you become more disciplined, but I just became very tired (laughs). I have six-year-old twins and the energy is not there anymore. I thought it would come back when they started sleeping through the night, but then age and genetics kicked in. But I now allow myself to say honestly which things are worth my time and which are not. I am not saying that playing with the kids is the ultimate activity to compare everything to – but I prioritize projects, people, and activities that are worth it given my limited energy and resources.

With very rare exceptions, I don’t work on weekends. In Europe, weekends are off limits. It is important to set an example for people in our groups. I don’t think anyone becomes more efficient or creative when overworked. Some people work best in bursts and then rest for several months, and that is fine as long as something good comes out in the end.

Belen: There has been a trend lately of senior people talking more about their failures and that is such a good thing. When I was a student, I felt like I only heard about successes and thought I was the only one getting papers rejected. It can be hard. I think it is our responsibility as supervisors to talk about our failures and let the younger ones know that it is part of the process. I think all of us have had papers that were rejected from SIGGRAPH and ended up published elsewhere. I have more rejected papers and grants than accepted ones.

Olga: All of my first-authored PhD papers – my "baby" projects – were all rejected from SIGGRAPH. Only my co-authored papers were at SIGGRAPH. For me, getting a paper accepted to SIGGRAPH is still a wonder. In my day, it was “Ooo, you have a SIGGRAPH paper!” Now it’s, “How many? Only 2?” So failing was more the norm.

I also believe that there is value in your advisor and peers believing in you. My advisor and my labmates were very supportive and believed strongly in my project and my abilities, which helped me develop my confidence and resilience during my PhD. I admit it is very hard, and as an advisor I feel even more pain now when my students have papers rejected. I try to motivate them despite the disappointment.

I have also been reminded that, ultimately, a paper decision is just the opinion of 4 or 5 people. It is not necessarily statistically significant. Eventually, all papers generally get published – maybe it won’t be at SIGGRAPH or it’s delayed until the following year, but ultimately it works out.

Yuting: My colleague told me that if I don’t have a lot of papers rejected, it’s because I don’t have enough submissions. My advisor also told me that if she makes a mistake, she will immediately make a bigger mistake and forget about the first one. Making mistakes and having failures is part of life. It’s no big deal. To be honest, most of the time we are not capable of making a mistake big enough to be unrecoverable, even if we try. Put your failure in context, find the silver lining, learn something from it, and do better next time.