Michele Newman: Before coming to UW, I was working on a project using natural language processing — AI, essentially — to measure creativity in elementary school children. When ChatGPT came out, I was at the UW working with KidsTeam, a program where adults and kids co-create technology products for children, and I really wanted to see what effects GPT might have on children's creativity.

So much of the early experience around this new technology was fearful. People were saying, “Don't use it to teach, it's going to harm kids.” Many schools banned it. So part of the impetus of the project was trying to see what a medium stance looks like — where it's not harming or taking jobs. It's supporting and building meaningful experiences for kids. How can we look to the future and build ethical and meaningful practices around this technology?

MN: In KidsTeam the primary methodology is co-design, where kids are treated as equal partners when designing technologies. So one of our approaches was just putting the kids in front of technology — OpenAI’s ChatGPT and Dall-E, or Google's music generator Magenta — to see what they do. What are their considerations? Where are they getting frustrated? What does it mean to have a tool that can actually kind of do the creation for you? A lot of creativity research talks about how process is very important. That's kind of where the person’s individuality comes out. So we wanted to see the kids develop their creative processes.

We also gave the kids a more structured experience. It's one thing to just look at a piece of technology and say, “Here's what it can do.” It’s another thing to say, “Use this specific software to write a story.” In the sessions, we balanced the open-ended approach with more directed exploration and had kids use techniques like comic boarding, where they make comics about potential good and bad uses of AI.

MN: Maybe the most important and practical finding is how clearly these systems are not built for children. The kids might know a lot about, say, a video game like Genshin Impact. If the AI system doesn’t know anything about it, the kids might conclude they’re smarter than the system. So there’s a mismatch between what children are expecting these systems to be able to do and what they can do. This type of technology is generally built with adults in mind. Likewise, children's language just isn't the same as adults. Things like this really become an issue for kids trying to creatively express themselves.

The title of this paper is “I want it to talk like Darth Vader,” which is a quote from one of the kids. He was writing a story about Star Wars, and he turned to us and said, “I want it to talk like Darth Vader. I want it to be able to be customized.” He suggested that it would help him write a better Star Wars story. Obviously, you could prompt ChatGPT to talk like Darth Vader, and we helped walk him through that. But those aren't things that the kids necessarily understand right away. They need extra instruction around that. Children's creativity is unique. Because of their development and their experiences, they have different needs than adults do. They're still building and understanding social norms, and what it means to create.

MN: Yeah, when we asked about some typical things like cheating, the kids tended to reiterate things they’ve heard, that “I shouldn't use it to cheat.” But when we asked them about things like whether their friend should use AI to write a birthday card for them, they started to have really nuanced takes. Some started asking how much the friend is using it. Is it to write the whole card, or just to help? Every kid starts to have different ideas. So then we’re considering how to foster an individual child's expression.

We asked one 11-year-old how he’d feel if his favorite book series was written by AI instead of an author, and he said it would “dismantle” the joy of reading for him. We often don't think about kids having these deep, existential questions about what it means to be an artist. But they are. They’re asking whether they lose some authenticity when AI rather than a friend writes a birthday card. Over the course of the study, we saw them changing and developing as they used these systems. By the end, it was great to hear them saying things like, “I don't think this really expresses what I'm saying.”

But they started making certain types of adjustments to their creative process and their goals, which for me sometimes raised a red flag. Sometimes they’d add extra context to get it to do what they wanted. But other times they might try an idea and quickly say, “It's not working, so I'm just going to change the idea.” That’s a hard problem. But we can't just make systems that solve all these issues, because every kid’s process is different. Sometimes you do need to learn to give up on an idea. That can be part of the creative process. So the question with AI is how do you support kids and give them knowledge of their individual creative processes? Creativity is always happening in a larger context. The interaction is not just about inputting a prompt. It’s working iteratively with the system while being supported by peers and adults. And those networks of support make a meaningful experience with these systems much more likely.