Create video using frame sequence with img2img

[b1tfl0w]

Hello, I recently tried to process a sequence of frames taken from a video with this script, the problem is that they all come out too different from each other, I thought that using the same seed they would maintain a certain coherence, but that was not the case. Any ideas?

#!/bin/bash

# Directory paths
INPUT_DIR="~/Pictures/input_frames"

# Hardcoded parameters
PROMPT="alien cat, psychedelic"
NEGATIVE_PROMPT=""
STRENGTH=0.5
SEED=1
IMAGE_WIDTH=1024
IMAGE_HEIGHT=512
INFERENCE_STEPS=1
GUIDANCE_SCALE=1.0

cd ~/fastsdcpu
source env/bin/activate

# Loop through each file in the input directory
for FILE in "$INPUT_DIR"/*; do
  # Check if it's a regular file (and not a directory)
  if [ -f "$FILE" ]; then
    # Execute your command with the appropriate arguments
    python src/app.py --use_openvino --openvino_lcm_model_id rupeshs/sdxl-turbo-openvino-int8 --use_offline_model --usejpeg --img2img -f "$FILE" --strength "$STRENGTH" --seed "$SEED" --image_height "$IMAGE_HEIGHT" --image_width "$IMAGE_WIDTH" --inference_steps "$INFERENCE_STEPS" --guidance_scale "$GUIDANCE_SCALE" --negative_prompt "$NEGATIVE_PROMPT" --prompt "$PROMPT"

    echo "Processed $FILE"
  fi
done

echo "All images processed."

[b1tfl0w]

I guess the answer to the question might be very complicate, anyway by using instead the last generated image gives intresting results:

#!/bin/bash

# Directory paths
INPUT_DIR="$HOME/Pictures/input_frames"
OUTPUT_DIR="$HOME/Pictures/output_frames"
RESULTS_DIR="$HOME/fastsdcpu/results"

# Hardcoded parameters
PROMPT="alien cat"
NEGATIVE_PROMPT=""
IMAGE_HEIGHT=512
IMAGE_WIDTH=512
INFERENCE_STEPS=1
GUIDANCE_SCALE=1.0
STRENGTH=0.1
SEED=1

# Initialize previous output image path
input_image="$1"
previous_output_image=""
img=""

cd ~/fastsdcpu
source env/bin/activate

# Process each input image in the directory
while : ; do
	# Construct command to generate the next image
	cmd='python src/app.py --use_openvino --openvino_lcm_model_id rupeshs/sdxl-turbo-openvino-int8 --use_offline_model --usejpeg --img2img --strength "$STRENGTH" --seed "$SEED" --image_height "$IMAGE_HEIGHT" --image_width "$IMAGE_WIDTH" --inference_steps "$INFERENCE_STEPS" --guidance_scale "$GUIDANCE_SCALE" --negative_prompt "$NEGATIVE_PROMPT" --prompt "$PROMPT"'

	if [ -n "$previous_output_image" ]; then
		cmd+=" --file $previous_output_image"
	else
		cmd+=" --file $input_image"
	fi

  # Execute the command and capture output
  # cleaning input dir
  echo "removing $INPUT_DIR/*"
  rm "$INPUT_DIR"/*
  echo "evaluating command"
  eval "$cmd"
  echo "moving files from "$RESULTS_DIR"/* to "$INPUT_DIR"/."
  mv "$RESULTS_DIR"/* "$INPUT_DIR"/.
  echo "coping all files from "$INPUT_DIR"/* to "$OUTPUT_DIR"/."
  cp -r "$INPUT_DIR"/* "$OUTPUT_DIR"/.
  echo "cding in "$INPUT_DIR""
  cd "$INPUT_DIR"
  img=$(ls *.jpg)
  cd ~/fastsdcpu
  echo "assigned img as $img"
  # Update previous output image path
  previous_output_image="$OUTPUT_DIR/$img"
  echo "Output saved in $OUTPUT_DIR/$img"
done

# Copy the final output image to the desired directory
#cp $previous_output_image $OUTPUT_DIR/output.jpg

Then inside the output folder rename the images sequentially by date:

#!/bin/bash
cd "$1"
for f in *.jpg
do
    mv -n "$f" "$(date -r "$f" +"%Y%m%d_%H%M%S").jpg"
done

Then run this to create the gif using ImageMagick:
convert -loop 0 *.jpg aliencat.gif
or for less quality:
convert -limit memory 9MB -loop 0 -resize 300 *.jpg aliencat.gif

[rupeshs]

@b1tfl0w Thanks for sharing this