麻豆破解版

Resurrecting a voice and creating new memories through AI

June 14, 2024

What does it mean to be in possession of your voice, and what would it mean for someone else to possess it? How do you reconstruct someone who is no longer here, and how real would that version of them be? Would it matter how real they are? 

These are some of the many philosophical, ethical, and technological questions that artificial intelligence (AI) has brought to the fore and that Sarah Sweeney, associate professor and chair of art, is contending with through her long-term project 鈥淐onversations with My Deepfake Dad.鈥�

Sweeney鈥檚 father, a screenwriter who documented his work and life in tape recordings, died at the age of 44, when Sweeney was 17.  

鈥淲hen I turned 44, I had all these thoughts about what it was like to be the same age he was when he died,鈥� Sweeney says. 鈥淲as I at the same place in my life?鈥�  

Around that time, she also began to take an interest in deepfakes 鈥� media generated using AI to replace someone鈥檚 likeness 鈥� and the implications of such capabilities.  

鈥淎s I was looking through flash drives and found old movies of my dad when he was very young, I had this thought: What would happen if you could make new memories with someone who had died?鈥�

Candid photos of Sweeney's father

Sweeney contacted a company called Resemble AI and began working with the firm to create a clone of her father鈥檚 voice using its speech-to-speech software. 鈥淚t takes someone鈥檚 speech 鈥� we鈥檝e been calling them 鈥榯he puppeteer,鈥� the person who鈥檚 speaking 鈥� and maps my dad鈥檚 voice onto it,鈥� she explains.  

Over the next several months, she found six hours of her father鈥檚 recordings and worked with Resemble AI and her mother to think through the ethical and technological challenges of generating new conversations using her father鈥檚 voice. 鈥淚 think a big question in this project is, 鈥榃ho has the right to say that someone who is no longer with us can be used in a deepfake?鈥� I had to produce a death certificate and a letter of consent from my mother. 

As Sweeney moved forward with the project, she considered what the conversations with her father would look like. 鈥淚 started thinking about, when someone dies, where might those answers come from?鈥� she recalls. 鈥淥ne of the things I became really interested in is how we hold someone after they die and how we imagine they鈥檙e the only version of that person. But in fact, there are maybe 60 versions of that person in the world held by different people.鈥� 
 
She decided she would generate six audio conversations, each based on different archival materials, interviews, and research. Sweeney writes both sides of each 20-minute-long conversation. She then speaks her side and her dad鈥檚 deepfake voice speaks his.  

鈥淭he first dad is based on interviews with my mom and my sisters. All his answers are written from what they think he would have said. So I got to meet my mom鈥檚 version of my dad and my sisters鈥� version of my dad. And I鈥檓 learning a lot about him through that.鈥�  

鈥淭he second one is just him, so that one is really interesting. It鈥檚 only words that he used in his letters and his tapes.鈥� 

鈥淭he third one is therapy sessions. They鈥檙e called empty chair therapy sessions. The fourth is a medium. The fifth is a chatbot. And the sixth is just voice memos to myself that I鈥檝e been recording in the car.鈥� 

The audio recordings, along with photographs, will constitute an online presence for her father.

From left, Science Machine Shop Supervisor/Technician Jeff Britton, Senior Teaching Professor of Physics Evan Halstead, Zo毛 Bilodeau 鈥�23, Omar Gomez 鈥�23, and Associate Professor of Art Sarah Sweeney work together to install the phone booth in Scribner Library.

Sweeney has taken the audio component to several classes at 麻豆破解版, including psychology, English, media and film studies, and First-Year Experience classes.  

鈥淚 listen to this file with them and then we have a conversation.鈥�  

Sweeney is keenly aware of the complex range of issues that her project raises, and she is one of a number of 麻豆破解版 faculty members 鈥� from Associate Professor of Psychology Jess Sullivan, who taught the course Artificial Self in spring 2023, to Professor of English Mason Stokes, who taught the course Writing with AI in fall 2023 鈥� who are grappling with the complexity of AI through their research and teaching. 麻豆破解版鈥檚 Center for Leadership, Teaching, and Learning and the College鈥檚 Learning Experience Design and Digital Scholarship Support (LEDS) teams even organized a semester-long AI Think Tank devoted to exploring some of those issues this past fall.  

Sweeney鈥檚 project continues to unfold, and rapidly evolving technologies continue to raise ethical, philosophical, legal, and other questions. Sweeney hopes her project will be thought-provoking in a multitude of ways, to many different audiences.

鈥淚f you look at the news, the worry right now is that our world is changing such that we can鈥檛 figure out what鈥檚 real and not real. This piece is a little bit about saying, 鈥業s there something interesting in something that鈥檚 both real and not real?鈥� And I think that鈥檚 what I really want us to consider about these technologies that work with our memories. They鈥檙e kind of incredible but also really scary. And I think that鈥檚 what this piece is really about 鈥� is this incredible possibility of this strange conversation I鈥檓 having with my dad.

鈥淥ne of the things people often ask me is, 鈥楧oes it matter that it鈥檚 real or not real?鈥� And I don鈥檛 think it does. At some points it really doesn鈥檛 matter. It feels like my dad.鈥� 

Presented through a collaborative 麻豆破解版 effort

The 鈥淐onversations with My Deepfake Dad鈥� audio recordings that Associate Professor and Chair of Art Sarah Sweeney creates are being shared online as well as through physical exhibitions.  

The first conversation was presented through a phone booth, which debuted in October 2022 at CultureHub in New York City. A parallel piece 鈥� a rotary phone 鈥� was sent to Los Angeles, and the phone booth installation was brought to Scribner Library for the spring and summer of 2023.

A number of collaborators at 麻豆破解版 have helped to bring the installation to life. The Art Department鈥檚 shop supervisor and building safety coordinator, Paul Davis, constructed the phone booth.

Andrea Casey, art technology support coordinator and program coordinator, created the prints for inside. Senior Teaching Professor Evan Halstead and Science Machine Shop Supervisor/Technician Jeff Britton from the Physics Department, along with three of Halstead鈥檚 independent study students 鈥� Omar Gomez 鈥�23, Zo毛 Bilodeau 鈥�23, and David O鈥機onnell 鈥�23 鈥� assembled the electrical components that allow the conversation to start and stop with the opening of the phone booth鈥檚 doors.

鈥淲hat I鈥檝e learned is that by partnering with people who have this incredible expertise 鈥� students, faculty, staff 鈥� the piece is so much richer, and I could never have made this thing by myself,鈥� Sweeney says.  

To listen to the conversations, visit