They use artificial intelligence to make movies, and you can help
(CNN) — “Salt” resembles many science fiction films of the 1970s and early 1980s, with 35mm footage of space freighters and alien landscapes. But as backward as it sounds, the way it was created hints at what could be a new frontier for movie-making.
“Salt” is the brainchild of Fabian Stelzer. He is not a film director, but for the past few months he has relied heavily on artificial intelligence (AI) tools to create this series of short films, which he releases roughly every few weeks. On twitter.
Stelzer creates the images with imaging tools like Stable Diffusion, Midjourney, and DALL-E 2. The voices are mostly created with AI voice generation tools like Synthesia or Murf. And he uses GPT-3, a text generator, to help write the script.
There is also an element of audience participation. After each new installment, viewers can vote about what should happen next. Stelzer takes the results of these polls and incorporates them into the plot of future films, which he can craft more quickly than a traditional filmmaker, since he uses these AI tools.
“In my little home studio I can make a ’70s sci-fi movie if I want to,” Stelzer, who lives in Berlin, said in an interview with CNN Business from that studio. “And I can actually make more than just a sci-fi movie. I can think, ‘What’s the movie in this paradigm, where the execution is as easy as the idea?'”
The plot is, at least for now, still vague. Like it shows the trailergenerally centers on a distant planet, Kaplan 3, where an overabundance of what initially appears to be mineral salt leads to dangerous situations, like somehow endangering an approaching spacecraft. To make things more confusing (and intriguing), different narrative threads and perhaps even some temporal anomalies are also introduced.
The resulting shorts are beautiful, mysterious and ominous. So far, each movie is less than two minutes long, in line with the maximum video length on Twitter, which is two minutes and 20 seconds. From time to time, Stelzer tweet a still image and a caption that contribute to the strange mythology of the series.
Just as AI imagers have already baffled some artists, Stelzer’s experiment provides a first example of how disruptive AI systems could be to filmmaking. As AI tools that can produce images, text, and voices become more powerful and accessible, they could change the way we think about generating and executing ideas. challenging what it means create and be a creator. Although the following for these videos is limited, some in the tech space are watching closely and hoping there will be more.
“Right now it’s in an embryonic phase, but I have a whole series of ideas of where I want to take this,” said Stelzer.
“Shadows of Ideas and Seeds of Stories”
The idea for “Salt” grew out of Stelzer’s experiments with Midjourney, a powerful, publicly accessible AI system where users can enter text and get an image in response. The prompts he gave the system generated images that he said “looked like a movie world,” showing things like alien vegetation, a mysterious figure lurking in the shadows, and a strange-looking research station on a barren mining planet. One of the images included what appeared to be salt crystals, he explained.
“I saw this in front of me and I was like, ‘Okay, I don’t know what’s going on in this world, but I know there are a lot of stories, interesting things,'” he said. “I saw narrative nuances and shadows of ideas and seeds of stories.”
Stelzer has a background in AI — he co-founded a company called EyeQuant in 2009 that was sold in 2018. But he doesn’t know much about making movies, so he started teaching himself with software and created a trailer for “Salt,” which tweeted on June 14 without any written introduction. (Of course, the tweet included an emoji of a salt shaker).
That was followed by what Stelzer calls the first episode a couple of days later. So far she has posted several, along with numerous still images and a few short film clips. Eventually, she hopes to turn the “Salt” pieces into a feature film, she said, and is creating a related company to make AI movies. She says that it takes him half a day to make each film.
The vintage sci-fi vibe is partly an homage to a genre Stelzer likes, and partly a necessity due to the technical limits of AI imagers, which still aren’t very good at producing images. with high fidelity textures. To get the AI to generate the images, he crafts prompts that include phrases like “a sci-fi research outpost near a mining cave,” “35mm footage,” “dark beige atmosphere,” and “salt crusts on the surface.” wall”.
The look of the film also fits Stelzer’s editing style as an amateur auteur. Since he uses AI to generate still images for “Salt,” Stelzer employs some simple techniques to make scenes appear animated, such as shaking parts of an image to make it appear to move or zooming in and out. It’s crude, but effective.
“Salt” goes to college
“Salt” has a small but charming following online. As of Wednesday, the film series’ Twitter account had some 4,500 followers. Some of them have asked Stelzer to show them how he makes his movies, he said.
Savannah Niles, director of product and design at augmented reality and virtual reality experience builder Magnopus, has followed “Salt” on Twitter and said she sees it as a prototype of the future of storytelling: when people actively participate and contribute to a narrative that AI helps build. She hopes that tools like the ones Stelzer uses can make film production cheaper and faster, which today can involve hundreds of people, take several years and cost millions of dollars.
“I think a lot of them are going to come out, which is exciting,” he says.
It is also used as a teaching aid. David Gunkel, a Northern Illinois University professor who has been watching the movies via Twitter, said he has previously used a short science fiction film called “Sunspring” to teach his students about computational creativity. Released in 2016 and starring “Silicon Valley” actor Thomas Middleditch, it is believed to be the first film to use AI to write its script. Now, he plans to use “Salt” in his fall semester communication technology classes, he said.
“Create a world in which you feel engaged, immersed,” he said. “I just want to see more of what’s possible, and what’s going to come out of it.”
Stelzer says he has a “somewhat cohesive” idea of what “Salt’s” overall narrative structure will be, but isn’t sure he wants to reveal it, in part because community involvement has already caused the story to deviate in some respects from what he had planned.
“Actually, I’m not sure that the story I have in mind will play out like this,” he said. “And the allure of the experiment for me, intellectually, is driven by curiosity to see what I, as a creator, and the community can come up with together.”