Black Mixture
patreon
FASTEST VIDEO GEN 🤯│LTXV Video Generation in ComfyUI: Complete Setup & Workflow Guide
🕑 Added 2024-11-24 10:54:06 +0000 UTCHello Black Mixture Fam! 👋
We're excited to dive into the latest addition to our toolkit: LTXV video generation! Day one support just dropped for the brand new model LTXV from Lightricks, and I've been testing it extensively to bring you this brand new workflow and comprehensive guide. Right now this is the FASTEST local video generator, allowing you to produce 2 second/24fps videos in roughly 8 seconds on an RTX 4090. Compared to previous models like CogVideoX or Mochi which would take roughly 10 - 20 minutes per generation, Lightricks' new LTXV model is insane!
📦 Here's what you'll need:
Main Components 🔧
Main Checkpoint Model
Download from: huggingface.co/Lightricks/LTX-Video
Place in: ComfyUI\models\checkpoints
CLIP Model 🎯
Get it from: huggingface.co/comfyanonymous/flux_text_encoders
Place in: ComfyUI\models\clip
Essential Workflows
We've created two optimized workflows that cover the most common video generation needs. As with all of our workflows, start from left to right.
Step 1. Load Models & Images
Step 2. Type in your Prompt (You can leave the default negative prompt)
Step 3. Name your project and video output.
1. 📝Text-to-Video Workflow
This is our go-to for creating videos from text prompts. The more detailed, the better the results.
2. 🖼️ Image-to-Video Workflow
Perfect for bringing your existing images to life or creating consistent style animations.
Key Features
Clean, optimized, & color-coded node setup
Built-in best recommended settings
Customizable video length and dimensions
Examples + Tips
Text-to-Video:
A man walks towards a window, looks out, and then turns around...A man walks towards a window, looks out, and then turns around. He has short, dark hair, dark skin, and is wearing a brown coat over a red and gray scarf. He walks from left to right towards a window, his gaze fixed on something outside. The camera follows him from behind at a medium distance. The room is brightly lit, with white walls and a large window covered by a white curtain. As he approaches the window, he turns his head slightly to the left, then back to the right. He then turns his entire body to the right, facing the window. The camera remains stationary as he stands in front of the window. The scene is captured in real-life footage.
Image-to-Video:
4k, high quality, house burning, on fire, smoke rises
General tips:
The model works on resolutions that are divisible by 32 and number of frames that are divisible by 8 + 1 (e.g. 257). In case the resolution or number of frames are not divisible by 32 or 8 + 1, the input will be padded with -1 and then cropped to the desired resolution and number of frames.
The model works best on resolutions under 720 x 1280 and number of frames below 257.
Prompts should be in English. The more elaborate the better. Good prompt looks like
The turquoise waves crash against the dark, jagged rocks of the shore, sending white foam spraying into the air. The scene is dominated by the stark contrast between the bright blue water and the dark, almost black rocks. The water is a clear, turquoise color, and the waves are capped with white foam. The rocks are dark and jagged, and they are covered in patches of green moss. The shore is lined with lush green vegetation, including trees and bushes. In the background, there are rolling hills covered in dense forest. The sky is cloudy, and the light is dim.
Comments
Black Mixture
Oh for sure it would be needing to update ComfyUI. The older versions of comfyui don't support the newest models yet. You might not need to update the python_dependencies, but you would need to update comfyui to the latest stabld build.
Black Mixture
Can you check the terminal for any errors related to installing the LTXV nodes when you first start ComfyUI and send them here? I might be able to diagnose a solution with more information. On another note, thanks for letting me know and sorry to hear it's not working. At the moment CogVideoX is still much better in terms of quality, but LTXV is incredibly fast when it works. There's a conflict though with the python dependencies used for the nodes with CogVideoX and LTXV, where some have seen that if you have LTXV working, CogVideoX ends up broken or vice versa. In the meantime, another solution some have used to get the LTXV nodes to install properly, is to try: 1. Go to your ComfyUI install and navigate to \ComfyUI_windows_portable\update 2. Run the update_comfyui_and_python_dependencies.bat file 3. Run the update_comfyui_stable.bat If this still doesn't work, a less than ideal solution is to have a separate fresh install of ComfyUI on a different drive without any pre-existing custom nodes and install LTXV first.
Alexander Pereira
The ComfyUI LTX node seems to not be recognized by the Manager as a missing node yet (when I load up the workflows). I gave it a few attempts to install anyways and did not manage to get it working, it was a failed installation. I tried with a ComfyUI installation using Python 3.11.9 and another with 3.12.7. I wonder if it has to do with Cuda versions, or something else entirely. At any rate, I wish these video models would let us generate 9:16 videos, as my use case is mostly Social Media.. I was just going to try this out of curiosity. I guess I'll stick with slow but reliable CogVideoX for playing around with local open source video.
