You’ve seen the Sora samples. The Dream Machine videos. How does LTX Studio, touted as “the future of storytelling, transforming imagination into reality,” stand up?
“There are whole bunch of things it does not do, but I love where it’s going and where I hope it’s going to go…. It’s brilliant for keeping track of all of the shots that you really do need to keep track of. It’s brilliant for scene wide settings and project wide settings, something I’ve been craving, and it’s really, really good at that. It’s great for casting. It’s brilliant for allowing you to then kind of just drop those characters in. I love the generative tools that will allow you to erase bits that you don’t need in your starting shot and to add other bits that you need that will help you tidy up the shot…. My two big gripes and I don’t think these are bugs that they’re going to fix, this is just fundamental features that it needs to be in there. One of them is every shot is slow motion…. Secondly, breaking the fourth wall. It drives me out of my mind!”
Here’s a peek at actually using LTX Studio by Riley Brown:
My take: In addition to Haydn’s slo mo and fourth wall gripes, I would add these requirements as well: movement and expression control including blinking and lip-sync. Mid-2024, one has to use each of the many AI tools for what it does best and then bring all the bits together in post. As an early proponent of Machinima (using video games to make movies,) I’m watching this space with interest. My conclusion: advances are being made but we’re nowhere near lucid dreaming.
Record in ProRes Log on an iPhone 15 Pro at 4K 30. He says, “4K 24 especially at ProRes Log just looks kind of choppy.”
In Settings, change to “Most Compatible” from “High Efficiency” and lock the white balance.
Turn on Exposure Adjustment and set it to .7. He says, “If your highlights are blown out it’s going to be a lot harder to actually bring that detail back once you go into color grading.”
Use the main 1X lens under adequate lighting, avoiding top-down noon sunlight. Try angling the light on the opposite side of the subject. He says, “The lighting is probably the most important element actually in making those cameras look good.”
Use the Grid to help create interesting compositions and make sure your camera movement is motivated.
In the edit, convert the Log footage with a Color Space Transform into Rec 709 and colour grade as usual.
Use Halation to lend the footage the characteristic film highlights glow and use a plugin called RSMB to add motion blur.
Hosts Marina Hyde and Richard Osman “pull back the curtain on television, film, journalism and more using their years of knowledge, enviable contact book and wit to bring what’s hot and what’s not in the world of entertainment.”
The episode has them reacting to the first Sora announcement and AI in general. This exchange is particularly insightful:
RO: “…that entire middle of the industry which is where everyone I’ve, you know, worked with my whole life works, you know, it’s going to be really, really tough times I think and there’s no safety net put in place but it’s the middle of culture that will disappear and be replaced by AI I suspect and it’s coming soon and it’ll come sooner.”
MH: “As always the middle in everything is going in the way that mid budget films have fallen away and that kind of middle bankable quality stuff has gone and at the moment I cannot see a pathway to it coming back.”
Richard also says:
“If you’re a new creator in the industry, if you’re a new brain (and by the way it’s quite hard to get into the industry) and you have these tools at your disposal, you would just be using them. I remember a very, very young Edgar Wright when he was starting out and him and his friends would just make these little home movies on videos because that was a new technology that they could suddenly use. If you’re a young person now, a young writer, young creator, young director, a young actor, why would you not be using these AI tools to make incredible content and sharing it with people?”
My take: These comments echo what Jerry Seinfeld recently said about the movie business in general being over. I foresee a continued splintering of audiences into smaller and smaller niches, necessitating smaller budgets as profits also become smaller. Every once in a while something will break through into the general Big C Culture but for the most part that will be left to deep-pocketed streamers chasing smaller and smaller mass audiences. As Seth Godin says:
“Based in Toronto, shy kids are a multimedia production company who utilized Sora for their short film about a balloon man. ‘We now have the ability to expand on stories we once thought impossible,’ shares the trio made up of Walter Woodman, Sidney Leeder and Patrick Cederberg. Walter, who directed Air Head, remarks that ‘as great as Sora is at generating things that appear real, what excites us is its ability to make things that are totally surreal. A new era of abstract expressionism.’ Speaking to the wider industry, ‘people from all over the world with stories ready to burst out of their chests finally have the opportunity to show the world what’s inside.'”
The BTS video is quite illustrative, as it does look like some post work was done on at least one clip:
I love this quote: “Sora is very much a slot machine as to what you get back.”
By the way, these are the folks behind 2013’s brilliant, genre-launching Noah (nsfw):
My take: sure, it’s not prime time yet, but Sora will only get better. Check out these other “text to video” tools on Future Tools. Oh, and I think the contrast in “air head” is too flat.
He begins, “This morning, a really lovely NFS reader reached out to me and wanted my opinion on a program called ScriptReader.AI, which charges $10 for a complete screenplay breakdown.”
He uploads one of his scripts and in two hours gets extensive coverage. It appears the AI’s method is to rate each scene and then average the scores. His script gets a “Recommend.”
However, Jason is not impressed. He concludes:
“Computers cannot think and feel. They have zero sense of nuance. And you cannot take their coverage seriously. No matter how nice it is. AI coverage is worthless!”
My take: the ScriptReader.ai website woefully undersells its capabilities. It certainly looks sketchy. But you can test out three scenes for free.
And then there’s a link in the top right corner of the website labelled “SHALL WE PLAY A GAME?_” I recognized this as a reference to WarGames in which Matthew Broderick unwittingly provokes WWIII.
With some trepidation, I clicked on the link.
This brought me to a page titled “SceneProv_” with two modes, Fun and Training.
I decided to have some fun and generate a scene for a RomCom. The AI, suddenly my writing partner, set the scene and populated it with two characters. My interactions are bolded.
“Genre selected: romance
INT. COFFEE SHOP – DAY
JEN sits at a table with a book, sipping on her latte. She notices AIDEN walk in and their eyes meet.
JEN:
(smiling)
Hey!
AIDEN:
(nervously)
Hi! Can I sit here?
JEN:
(surprised)
Uh, sure!
AIDEN sits across from Jen and they strike up a conversation.
AIDEN: I noticed you’re reading “Do Androids Dream of Sheep?”.
JEN: Yup. It’s my favourite book.
Her eyes glitch.
AIDEN: Do you want to go back to my place?
JEN:
(surprised)
Excuse me?
AIDEN:
(smiling)
Not like that. I have a first edition copy of that book. I’d love to show it to you.
JEN:
(relieved)
Oh, sure. That sounds great.
INT. AIDEN’S PLACE – NIGHT
Aiden and Jen are sitting on the couch, looking at the first edition. Her eyes glitch again.
AIDEN Jen, why do your eyes “glitch?”
JEN What do you mean?
AIDEN Well, every once in a while they kinda spark.
Are you hiding something from me?
Jen looks at Aiden angrily and stands up quickly.
JEN:
(angrily)
No, I’m not hiding anything! It’s just a glitch in my synthetic retinas. I’m an android.
AIDEN:
(surprised)
An android? I had no idea.
JEN:
(sadly)
I know, it’s not exactly a desirable trait in the dating world.
AIDEN:
(smiling)
Well, I’m not like most guys. I think it’s pretty cool, to be honest.
Jen’s eyes glitch again, this time in surprise.
JEN:
(surprised)
Really?
AIDEN:
(nodding)
Really. Plus, I always thought I’d make a great cyborg myself.
Jen smiles and takes Aiden’s hand, feeling grateful for the first time to have found someone who accepts her for who she really is.
They continue to look at the book, their fingers touching.
Aiden turns to Jen, looking into her eyes.
AIDEN:
(softly)
You know, I may not be able to dream like you do, but
I have a feeling this is the beginning of something special.
JEN:
(smiling)
I have that feeling, too.
They lean in and share a kiss, the start of their unconventional love story.”
And then it rated “our” scene a B+. This AI passes the Turing Test for sure.
The inevitability of script-to-screen technology is closer than ever.
OpenAI released test footage and announced, “Introducing Sora, our text-to-video model. All the clips in this video were generated directly by Sora without modification. Sora can create videos of up to 60 seconds featuring highly detailed scenes, complex camera motion, and multiple characters with vibrant emotions.”
“Sora is able to generate complex scenes with multiple characters, specific types of motion, and accurate details of the subject and background. The model understands not only what the user has asked for in the prompt, but also how those things exist in the physical world.”
“The model has a deep understanding of language, enabling it to accurately interpret prompts and generate compelling characters that express vibrant emotions. Sora can also create multiple shots within a single generated video that accurately persist characters and visual style.”
Beyond text-to-video, “Sora can also be prompted with other inputs, such as pre-existing images or video. This capability enables Sora to perform a wide range of image and video editing tasks — creating perfectly looping video, animating static images, extending videos forwards or backwards in time, etc.”
Sora can even replace the whole background in a video: “Diffusion models have enabled a plethora of methods for editing images and videos from text prompts…. One of these methods, SDEdit,32… enables Sora to transform the styles and environments of input videos zero-shot.”
My take: this is powerful stuff! Workers in media industries might want to start thinking about diversifying their skills….
“Pulling from our friends at B&H, it looks like their crew featuring Nick and Dave were able to take the new Kodak Super 8 camera through the paces as they ran around New York City.”
My take: This is way too expensive for me! And the image still bounces around like crazy. IMHO Kodak missed an opportunity to improve their technology by bypassing the cartridge’s internal pressure plate and figuring out how to stabilize each frame during exposure, kinda like the “true pin registration” that the Logmar GENTOO GS8 does. Pity.
“Film Rescue has a global reach — and reputation — with a collection office in the Netherlands and another in northern Montana. Every few weeks, a batch of 200 to 400 rolls of old camera and home movie film appears at the Indian Head office, which is housed in a 125-year-old bank building on the main highway through town.”
“The film rescuers consist of Miller, Gostick and five employees…. One employee, Gerald Freyer, is a European-trained digitization expert. He was beavering away for a German museum when Miller recruited him to move to the Prairies. His latest star acquisition is Heather Harkins, who trained at the renowned Selznick School of Film Preservation in Rochester, N.Y., and was doing contract work for museums and archives throughout North America when she got a cold-call job offer.”
