IMAX AI is born ?
On November 30, 2022, ChatGPT took the world by storm, showcasing the power of AI in our daily lives. It rapidly became a valuable tool across various industries for simplifying tasks that previously required manual searches. However, it wasn’t all smooth sailing. Users soon noticed biases, particularly leaning towards leftist views, inaccuracies, and availability issues. Websites then adopted a “prompt-interface” that relied on ChatGPT’s core technology, making it seem like AI was everywhere. Another significant player, Midjourney, an advanced image generation AI, gained popularity for image upscaling via Discord. Bing introduced its AI chat model, incorporating ChatGPT4, making rapid answers more accessible. Now, let’s connect this AI phenomenon to IMAX and the creator of Star Wars. IMAX provides a cinematic experience close to reality due to its 15/70 format, characterized by a 70mm celluloid and 15 sprocket holes on each film frame.
But how does The Creator claim IMAX quality from shooting with Sony’s FX3 in 4K?
Well, if you’re even a bit familiar with IMAX, then you know that it delivers close to life-like cinema experiences. Imax is in cinema as good as it gets. But, one has to wonder – how did Imax even get here ? True Imax is 15/70 – which is defined by the size of Celluloid (70mm), and 15 sprocket holes on a single frame of film – yes – that is physical film. Imax has been around for a while (since 1970’s to be exact), and even though Vistavision (35mm) existed for a while, one doesn’t need lots of math to understand that 65mm is almost double. Project it at 70mm and you do actually get double the size. Besides being double the size it is worth noting that Super35 (actually smaller then 35mm) has been a film prefered format due to it’s smaller size (being easier on camera operators – as in lighter) as well as cheaper due to it’s size.
How does The Creator fit in here ?
Despite being shot on the FX3, The Creator boasts IMAX Certified Camera status. However, finding solid data on Sony’s FX3 being IMAX certified is a bit elusive. While an IMAX film scan can reach a remarkable 19K resolution, how can we truly achieve an IMAX experience from a 4K source? A few months ago, we worked on a custom algorithm that harnessed AI to randomize noise patterns, upscale images, and introduce a shutter effect to mimic large-scale celluloid. Back then, our raw footage was captured on a GH4, which maxed out at 4096×2160 resolution and featured 4:2:0 8-bit capabilities. Despite access to more expensive gear, we chose to keep Reel Goldstar projects in-house. Simple resizing wouldn’t suffice, and we noticed that after a 30-40% upscale, the raw footage lost its 4K quality. Our approach involved spatial versus temporal analysis, tracking data, substituting new data for old, and generating a point cloud of pixels to create a 3D projection mesh. While it enhanced temporal resolution, it remained limited to 4-6K outputs due to its intensive computational demands.
The idea wasn’t entirely our own; it’s akin to image stacking used in astrophotography or Panasonic’s advanced cameras that achieve higher resolution by shifting sensors and capturing the same image repeatedly. However, we found this approach demanding excessive computing power for a “marginal” improvement.
So, how did they manage to achieve the IMAX experience?
They began by shooting the movie in a 2.76:1 ultra-wide aspect ratio, already coming “close” to the IMAX size. For reference, the Arri Alexa 65 closely mirrors the digital IMAX standard. By utilizing a 2x Anamorphic, they compressed horizontal space on the camera sensor, contributing to their success.
When the frame is compressed, we achieve a significantly wider image, nearly double the size of Sony’s full-frame sensor. Utilizing a straightforward image upscaling algorithm, we realize that the concept isn’t as far-fetched as it initially appeared.
A quick online search reveals numerous image upscaling tools, with Topaz Video AI being a prominent choice. While it delivers impressive results, it comes with a price tag that may be a non-factor in feature film production, pushing UHD images closer to 8K or even 12K, not far from the 19K benchmark.
While we’re not endorsed by Blackmagic, we recommend their 12K Ursa camera for those daring to attempt a similar endeavor.
But, the journey doesn’t end here. Even though we conducted our research and developed our own tool (albeit with limited utility), it’s evident that technology has taken us further than we could have imagined. This exploration will persist, and for now, we share this valuable insight.
Update: After witnessing the Creator in 4K Quality, it was evident that the unique method to simulate film stock was available for some time.
The FX3 is a insanely clean camera, even at higher ISO settings, thus using it for that raw,gritty, organic celluloid emulation out of the box is really impossible. And though the story was really amazing to enjoy, I took some specific time to research out under the surface.
The film was shot on FX3 cameras, despite what everyone says tha it was (some claim it was a marketing stunt and it was a Alexa 35) – not likely.
Anamorphic KOWA Lenses and being shot on Atomos Ninja (using a raw stream out of Fx3 in ProRes RAW format proved to be more then enough for IMAX standard, as well as vast Visual Effects used for the film.
Anto
