What is Runway?
Runway is a powerful AI tool that transforms text inputs into high-quality video clips. It offers a range of creative possibilities, from generating videos based on text descriptions to animating images into dynamic footage. Runway’s advanced models make it a go-to for artists, filmmakers, and content creators looking to leverage AI in their projects.
Runway Gen 2 vs Gen 3 Alpha
Runway Gen 3 Alpha marks a significant upgrade from Gen 2. It offers major improvements in fidelity, consistency, and motion, providing more lifelike and coherent video outputs. Trained on both videos and images, Gen 3 Alpha enhances Runway’s capabilities across various tools:
- Text to Video: Creates videos directly from text prompts.
- Image to Video: Animates static images into videos.
- Text to Image: Generates images from text descriptions.
- Motion Brush, Advanced Camera Controls, and Director Mode: Allows for more precise control over video elements such as structure, style, and motion.
Prompt Modes in Runway
Runway Gen 2 offers three primary ways to generate content. At this time, Gen 3 Alpha only offers text to video prompts, but that will most likely expand. Here are the 3 prompt modes offered in Gen 2.
Text Only
This mode is best for creating videos purely from text descriptions. It generates footage that matches the given prompt, ideal for storytelling or conceptual videos.
Image Only
In this mode, you can upload an image, and Runway will animate it into a video. This is useful when you want the final video to closely resemble the provided image.
Image and Text Description
Combining both image and text, this mode uses the image as a base and the text to add context and detail, resulting in a video that aligns closely with both inputs.
How to Prompt with Runway
Avoid Conversational Prompts
Unlike other AI services, Runway does not respond well to conversational prompts. Instead, focus on clear, descriptive visuals. Avoid phrases that resemble dialogue or commands, as they can confuse the model.
Descriptive, Not Command-Based
Runway models generate visuals based on descriptions, not commands. Don’t write a prompt like: “Add a flower to this clip of a meadow.” or “Make the horse jump through the hoop.” Instead of instructing the model to “add” or “make” something happen, describe the scene or object you envision.
One Visual Item Per Prompt
Each prompt should focus on one visual item or scene. Don’t expect a full story or sequence. A single prompt usually generates one shot or image, so keep your descriptions concise and specific to one element. Occasionally, you’ll get an output that radically changes scenes in the middle of a shot, which may be mistaken for a cut, but the shot will most likely be unusable (unless you like that surreal aesthetic)
Technical Details Can Enhance Results
If you have a background in film production, use technical terms related to lighting, camera specifications, and lens effects. Words like “cinema noir” or “volumetric lighting“, “anamorphic lens”, or even “8K” can significantly enhance the quality of your generated videos.
Structuring Runway Gen 3 Alpha Prompts
To get the best results with Runway Gen 3 Alpha, Runway recommends that you structure your prompts in specific manner. Use the following format:
[camera details]: [establishing scene]. [additional details]
- Camera Details: Specify the camera angle, shot type, or movement.
- Establishing Scene: Describe the main subject and setting.
- Additional Details: Add any extra information about the scene, subjects, and movement style.
Examples
By following these guidelines and structuring your prompts effectively, you can leverage Runway’s capabilities to create high-quality video outputs. You can also use Word.Studios Runway Prompt Builder to quickly create prompts by filling in a few fields.
