New Generated Video pipeline?

A couple of very recent videos point to a potential new Generated Video, or GV, pipeline.

The first is “Create Cinematic Ai Videos with Kling Ai! – Ultra Realistic Results” by Seattle’s Yutao Han, aka Tao Prompts.

The second is “How-To Create Uncensored Images Of Anyone (Free)” by Lisbon’s Igor Pogany, aka The AI Advantage.

Imagine combining both into a new GV pipeline:

  1. Train custom character models
  2. Create key frames utilizing these custom models
  3. Animate clips with these key frames
  4. Upscale these clips
  5. Edit together.

My take: a lot of people will immediately claim this is heresy, and threatens the very foundations of cinema as we’ve come to know it over the last one hundred years. And they would be right. And yet, time marches on. I believe some variation of this is the future of ultra-low budget production. Very soon the quality will surpass the shoddy CGI that many multi-million dollar Hollywood productions have been foisting on us lately.

August 2024 AI Video Pipeline

Love it or hate it, as of August 2024, AI Video still has a long way to go.

In this video, AI Samson lays out the current AI Video Pipeline. Although there are a few fledgling story-building tools in development, full-featured “story mode” is not yet available in AI video generators. The current pipeline is:

  1. Create the first and last frames of your clips
  2. Animate the clips between these frames
  3. Create audio and lip-sync the clips
  4. Upscale the clips
  5. Create music and SFX
  6. Edit everything together offline.

It seems new platforms emerge weekly but AI Samson makes these recommendations:

00:23 AI Art Image Generators
09:19 AI Video Generators
16:28 Voice Generators
18:02 Music Generators
20:44 Lip-Syncing
21:52 Upscaling

Keep an eye open for LTX Studio though.

My take: You know, the current pipeline makes me think of an animation pipeline. It’s eerily similar to the Machinima pipeline I used to create films in the sandbox mode of the video game The Movies over ten years ago:

Paris 2024 Opening Ceremony was spectacular!

The Opening Ceremonies of the Paris 2024 Olympic Games were four hours long and I enjoyed every minute!

Lady Gaga performed!

And Céline Dion closed the show from the Eiffel Tower:

From the Olympics:

“For the first time in the history of the Olympic Summer Games, the Opening Ceremony will not take place in a stadium. The parade of athletes will be held on the Seine with boats for each national delegation. Winding their way from east to west, the 10,500 athletes will cross through the centre of Paris. The parade will come to the end of its 6-kilometre route in front of the Trocadéro, where the remaining elements of Olympic protocol and final shows will take place. Eighty giant screens and strategically placed speakers will allow everyone to enjoy the magical atmosphere of this show reverberating throughout the French capital.”

My take: It was spectacular! As it should be, costing a reported 120 million Euro.

July 2024 Tier List for AI Video

Igor Pogany of The AI Advantage recently released a YouTube video that succinctly summarizes the current state of AI Video.

The tools he reviews are:

His favourites (dated mid-July 2024) are:

Runway GEN-3 Alpha and Luma Dream Machine for their clip outputs, but watch out for LTX Studio because of their overall project approach.

See the full tier list at 12:48 for the tl;dr.

My take: this is a super-valuable video that can get you up-to-date in under 14 minutes. Well worth your time.

Reality check: LTX Studio mid-2024

You’ve seen the Sora samples. The Dream Machine videos. How does LTX Studio, touted as “the future of storytelling, transforming imagination into reality,” stand up?

Haydn Rushworth posted this review:

“There are whole bunch of things it does not do, but I love where it’s going and where I hope it’s going to go…. It’s brilliant for keeping track of all of the shots that you really do need to keep track of. It’s brilliant for scene wide settings and project wide settings, something I’ve been craving, and it’s really, really good at that. It’s great for casting. It’s brilliant for allowing you to then kind of just drop those characters in. I love the generative tools that will allow you to erase bits that you don’t need in your starting shot and to add other bits that you need that will help you tidy up the shot…. My two big gripes and I don’t think these are bugs that they’re going to fix, this is just fundamental features that it needs to be in there. One of them is every shot is slow motion…. Secondly, breaking the fourth wall. It drives me out of my mind!”

Note that LTX Studio can do lots of things:

  • Pitch Decks
  • Storyboards
  • Animatics
  • Videos

Check out the video at the bottom of the corporate webpage.

Here’s a peek at actually using LTX Studio by Riley Brown:

My take: In addition to Haydn’s slo mo and fourth wall gripes, I would add these requirements as well: movement and expression control including blinking and lip-sync. Mid-2024, one has to use each of the many AI tools for what it does best and then bring all the bits together in post. As an early proponent of Machinima (using video games to make movies,) I’m watching this space with interest. My conclusion: advances are being made but we’re nowhere near lucid dreaming.

Sex in Cinema in 2024

Stephen Follows claims in Why is sex in movies declining? that “Comparing each year to the baseline of 2000, we can see a steady decline in the amount of sex in feature films. By 2023, it had fallen by almost 40% from the start of the century.”

He says the reasons for this could be:

  1. There are fewer sexual scenes overall.
  2. There are more movies without any scenes of a sexual nature, particularly in action and thriller genres.
  3. Producers and filmmakers might be more sensitive to how sexual content could be perceived or potentially lead to controversy.
  4. Explicit sex scenes could result in more restrictive age ratings or censorship, thereby reducing a film’s international box office potential.
  5. There might be less demand for sexual content in wide-release films on family-friendly streaming platforms.
  6. There could be a rejection of outdated stereotypes as seen through a predominantly male gaze.
  7. The availability of more adult content elsewhere.

However, Jillian Angelini claims on Pop SugarSex Scenes Are So Back — and Better.

She says, “Now, explicit films such as “Poor Things,” “Love Lies Bleeding,” and “The Idea of You” are sending a clear message: Hollywood is hornier than ever.

She concludes:

Sex scenes are back, but not like they were before. Instead, they’re transforming to reflect desires for more respectful and authentic intimacy. This era suggests a promising shift towards more inclusive cinema where on-screen intimacy can feel empowering and enlightening for all.”

My take: People who know me know I love romance in movies. Conversely, I abhor violence. Make love, not war!

Is the digital ecosystem starting to look a lot like TV?

The Spring 2024 edition of the Canada Media Fund’s Perspectives is titled, “Embracing Change.”

Nicole Matiation of Nordicity posits:

Content is still king, but market imperatives and evolving technology are reshaping business models. It is indeed back to the future as streamers and tech giants incorporate both advertising and subscription revenue streams into their direct-to-consumer business model…. The digital platform ecosystem is starting to look a lot like conventional television.

Section 1 examines the shift towards a profitability model in the streaming industry. Major streaming platforms like Netflix, Disney+, and Prime Video are moving away from a subscriber growth-at-all-costs approach to focus on profitability. This includes introducing ad-supported tiers, bundling strategies, and emphasizing tentpole content releases to retain subscribers and boost revenue.

Section 2 analyzes audience behaviour, highlighting that while streaming is popular, linear TV remains relevant across age groups. Audiences still spend significant time watching traditional TV, prompting broadcasters and streaming giants to invest in live events, sports rights, and appointment viewing content to “win the living room.” It also explores the importance of second-screen engagement for content promotion.

Section 3 focuses on the video game industry. Seeing gaming as a new frontier for growth, major streamers like Netflix, Disney+, and YouTube are venturing into cloud gaming services and integrating video game content onto their platforms. While virtual reality (VR) and the metaverse faced setbacks in adoption, immersive experiences in gaming, extended reality (XR), and exhibitions are gaining traction, particularly among younger audiences. The industry is exploring ways to leverage this emerging space for storytelling and engagement.

Section 4 covers key factors shaping the industry’s future, including the impact of artificial intelligence (AI,) initiatives to reduce carbon footprints and the importance of demographic reporting and data collection efforts to promote equity, diversity, inclusion, and accessibility (EDIA) in the media landscape.

You can read the full PDF.

My take: I think this report is overly optimistic and contrasts with global media trends. In the UK, Nic Newman on Reuters Institute says advertising revenues remain unstable, social media referral traffic is declining, and there is a significant shift towards paid subscription models, all while AI integration and ethical concerns around it are becoming critical issues facing media outlets. In the U.S., media companies are grappling with similar challenges: “Consumers are questioning the value of streaming media while also declaring their unwillingness to ever pay for social media. Just as streaming video providers are rebuilding the ad models that buoyed pay TV, fewer people surveyed are moved by commercial advertising and, instead, seek recommendations from trusted creators and influencers to help them navigate and find value. More are turning to online multiplayer video games for virtual friendship, content discovery, and brand and franchise interactions.” Globally, according to Deloitte, “Convergence has really defined the industry in recent years, as the lines between gaming, TV, and film become even more blurred. User-generated video is competing for audience attention — and advertising dollars. Underpinning all these trends is the influence of diverse and tech-savvy younger generations, who are redefining media and entertainment as they move freely across TV, film, gaming, and social media.” I feel that overall, while there are pockets of growth and innovation, the global media landscape is marked by significant economic, technological, and trust-related challenges that contrast with the hopefulness presented in “Embracing Change.”

The “Netflix Tax” is very near

The CRTC has announced that streamers doing business in Canada will help pay for CanCon by contributing 5% of their revenues starting September 2024.

“The CRTC is requiring online streaming services making more than $25 million to contribute 5% of their Canadian revenues to support the Canadian broadcasting system.”

The 5% contributions from online streaming services will go to:

  • 2% to the Canada Media Fund and/or direct expenditures towards certified Canadian content;
  • 1.5% to the Independent Local News Fund;
  • 0.5% to the Black Screen Office Fund, the Canadian Independent Screen Fund for BPOC creators, and/or the Broadcasting Accessibility Fund;
  • 0.5% to the Certified Independent Production Funds supporting OLMC producers and producers from diverse communities; and
  • 0.5% to the Indigenous Screen Office Fund.

The 5% contributions from audio streaming services will go to:

  • 2% to FACTOR and Musicaction;
  • 1.5% to a new temporary fund supporting local news production by commercial radio stations outside of the designated markets;
  • 0.5% to the Canadian Starmaker Fund and Fonds RadioStar;
  • 0.5% to the Community Radio Fund of Canada;
  • 0.35% to direct expenditures targeting the development of Canadian and Indigenous content and/or a variety of selected funds; and
  • 0.15% to the Indigenous Music Office and a new fund to support Indigenous music.

The CRTC feels this decision balances the playing field in Canada:

“For decades, traditional Canadian television and radio services have financially supported the creation of content made by and for Canadians, and have showcased that content on their services. Meanwhile, online streaming services, which have been operating in Canada for well over a decade, have not been required to contribute in similar ways. In the Commission’s view, online undertakings that benefit from their place in the Canadian broadcasting system by generating significant revenues and drawing significant Canadian audiences should contribute to the system.”

There is a silver lining. Streamers can count their Canadian projects as 2% of their 5%. “Online streaming services will have some flexibility to direct parts of their contributions to support Canadian television content directly.”

My take: This 5% is not actually a tax that will be added to consumers’ bills. However, I’m almost positive streamers will raise their prices by at least 5%, so perhaps it’s a corporate levy, and not a consumer tax. Just get ready to pay more.

Apple Log Tips

Full Time Filmmaker on YouTube recently released “The Secret Formula for Cinematic iPhone Videos“.

In it he reveals the following:

  1. Record in ProRes Log on an iPhone 15 Pro at 4K 30. He says, “4K 24 especially at ProRes Log just looks kind of choppy.”
  2. In Settings, change to “Most Compatible” from “High Efficiency” and lock the white balance.
  3. Turn on Exposure Adjustment and set it to .7. He says, “If your highlights are blown out it’s going to be a lot harder to actually bring that detail back once you go into color grading.”
  4. Use the main 1X lens under adequate lighting, avoiding top-down noon sunlight. Try angling the light on the opposite side of the subject. He says, “The lighting is probably the most important element actually in making those cameras look good.”
  5. Use the Grid to help create interesting compositions and make sure your camera movement is motivated.
  6. In the edit, convert the Log footage with a Color Space Transform into Rec 709 and colour grade as usual.
  7. Use Halation to lend the footage the characteristic film highlights glow and use a plugin called RSMB to add motion blur.

An excellent video!

My take: Davinci Resolve 19 Studio has just come out with a Film Look Creator effect!

It seems like you’ll still want to add some motion blur, which you can do in Davinci Resolve:

The “middle” will disappear

The Rest Is Entertainment on Youtube recently released an episode with a take on How AI Will Take Over Hollywood In The Next 10 Years.

Hosts Marina Hyde and Richard Osman “pull back the curtain on television, film, journalism and more using their years of knowledge, enviable contact book and wit to bring what’s hot and what’s not in the world of entertainment.”

The episode has them reacting to the first Sora announcement and AI in general. This exchange is particularly insightful:

RO: “…that entire middle of the industry which is where everyone I’ve, you know, worked with my whole life works, you know, it’s going to be really, really tough times I think and there’s no safety net put in place but it’s the middle of culture that will disappear and be replaced by AI I suspect and it’s coming soon and it’ll come sooner.”

MH: “As always the middle in everything is going in the way that mid budget films have fallen away and that kind of middle bankable quality stuff has gone and at the moment I cannot see a pathway to it coming back.”

Richard also says:

“If you’re a new creator in the industry, if you’re a new brain (and by the way it’s quite hard to get into the industry) and you have these tools at your disposal, you would just be using them. I remember a very, very young Edgar Wright when he was starting out and him and his friends would just make these little home movies on videos because that was a new technology that they could suddenly use. If you’re a young person now, a young writer, young creator, young director, a young actor, why would you not be using these AI tools to make incredible content and sharing it with people?”

My take: These comments echo what Jerry Seinfeld recently said about the movie business in general being over. I foresee a continued splintering of audiences into smaller and smaller niches, necessitating smaller budgets as profits also become smaller. Every once in a while something will break through into the general Big C Culture but for the most part that will be left to deep-pocketed streamers chasing smaller and smaller mass audiences. As Seth Godin says: