Random 2x+ slower generations with the same settings #2434
-
|
First - this might be some user error caused by my lack of understanding. This is why I didn't created an issue, maybe is just a setting that I played with or maybe is something that happens to others too. After I write the prompts or use the "Process image" to transfer the prompts from a previous generated image I change to the correct sampling method and upscaler (needed after the update because the names of the upscales/samplers changed) - I press generate.
When it's slower I can see it staying longer right after completing the Upscaling part. 640x800 with 2x upscale. device: NVIDIA GeForce RTX 2080 (1) (sm_90) (7, 5) app: SD.next arch: AMD64 using Chrome || Backend original, 1.5 models I updated by doing a fresh/clean install with git clone https://github.com/vladmandic/automatic into a new location - still have the previous install on the same disk but other folder, I just moved the models, loras, etc. Please let me know if there is anything relevant that I can provide that I missed. There is no error/warning in the console. |
Beta Was this translation helpful? Give feedback.
Replies: 3 comments 9 replies
-
|
post |
Beta Was this translation helpful? Give feedback.
-
|
SLOW RUN 2023-11-03 17:48:16,532 | sd | DEBUG | txt2img | txt2img: id_task=task(a2zvp1kotcwpzs8)|prompt=21 year old beautiful Spanish model, brunette long messy hair, ( in a cave:1.3), bat, halloween, jack-o'-lantern, pumpkin, witch_hat, body_writing, thighhighs, spider_web, vampire_costume, bat_hair_ornament, graveyard, orange_ribbon, orange_choker, bat_wings, candle, orange_bow, open_mouth, bat_print, castle, orange_panties, black_legwear,vegetable, bow, shoulder_tattoo, gloves, twintails, spider_web_print, lantern, grave, charmander, orange_legwear, navel, orange_bowtie, elbow_gloves, dynamic pose, lora:weight_slider_v1:-1.2 lora:epiCRealismHelper:1.0, hourglass body shape, lora:hourglassv01:0.5, lora:1Miriam2:0.4, lora:StokeRealV1:0.6|negative_prompt=(big head, huge head:1.2), (bangs,:1.3) head out of frame, out of canvas, nipples, (hands:1.3), long neck, big head, asian, chinese, cartoon, painting, illustration, (worst quality, low quality, normal quality:2), (extra limbs:1.3)|prompt_styles=[]|steps=23|sampler_index=11|latent_index=11|full_quality=True|restore_faces=False|tiling=False|n_iter=1|batch_size=1|cfg_scale=5.8|clip_skip=1|seed=-1.0|subseed=-1.0|subseed_strength=0|seed_resize_from_h=0|seed_resize_from_w=0||height=800|width=640|enable_hr=True|denoising_strength=0.35|hr_scale=2|hr_upscaler=ESRGAN 4x_NMKD-Superscale-SP_178000_G|hr_force=True|hr_second_pass_steps=14|hr_resize_x=0|hr_resize_y=0|image_cfg_scale=6|diffusers_guidance_rescale=0.7|refiner_steps=5|refiner_start=0.8|refiner_prompt=|refiner_negative=|override_settings_texts=[] FAST RUN 2023-11-03 17:49:28,845 | sd | DEBUG | txt2img | txt2img: id_task=task(eehc6cnkshuivem)|prompt=21 year old beautiful Spanish model, brunette long messy hair, ( in a cave:1.3), bat, halloween, jack-o'-lantern, pumpkin, witch_hat, body_writing, thighhighs, spider_web, vampire_costume, bat_hair_ornament, graveyard, orange_ribbon, orange_choker, bat_wings, candle, orange_bow, open_mouth, bat_print, castle, orange_panties, black_legwear,vegetable, bow, shoulder_tattoo, gloves, twintails, spider_web_print, lantern, grave, charmander, orange_legwear, navel, orange_bowtie, elbow_gloves, dynamic pose, lora:weight_slider_v1:-1.2 lora:epiCRealismHelper:1.0, hourglass body shape, lora:hourglassv01:0.5, lora:1Miriam2:0.4, lora:StokeRealV1:0.6|negative_prompt=(big head, huge head:1.2), (bangs,:1.3) head out of frame, out of canvas, nipples, (hands:1.3), long neck, big head, asian, chinese, cartoon, painting, illustration, (worst quality, low quality, normal quality:2), (extra limbs:1.3)|prompt_styles=[]|steps=23|sampler_index=11|latent_index=11|full_quality=True|restore_faces=False|tiling=False|n_iter=1|batch_size=1|cfg_scale=5.8|clip_skip=1|seed=-1.0|subseed=-1.0|subseed_strength=0|seed_resize_from_h=0|seed_resize_from_w=0||height=800|width=640|enable_hr=True|denoising_strength=0.35|hr_scale=2|hr_upscaler=ESRGAN 4x_NMKD-Superscale-SP_178000_G|hr_force=True|hr_second_pass_steps=14|hr_resize_x=0|hr_resize_y=0|image_cfg_scale=6|diffusers_guidance_rescale=0.7|refiner_steps=5|refiner_start=0.8|refiner_prompt=|refiner_negative=|override_settings_texts=[] SLOW RUN 2023-11-03 17:51:13,612 | sd | DEBUG | txt2img | txt2img: id_task=task(monnnl1himvubpz)|prompt=21 year old beautiful Spanish model, brunette long messy hair, ( in a cave:1.3), bat, halloween, jack-o'-lantern, pumpkin, witch_hat, body_writing, thighhighs, spider_web, vampire_costume, bat_hair_ornament, graveyard, orange_ribbon, orange_choker, bat_wings, candle, orange_bow, open_mouth, bat_print, castle, orange_panties, black_legwear,vegetable, bow, shoulder_tattoo, gloves, twintails, spider_web_print, lantern, grave, charmander, orange_legwear, navel, orange_bowtie, elbow_gloves, dynamic pose, lora:weight_slider_v1:-1.2 lora:epiCRealismHelper:1.0, hourglass body shape, lora:hourglassv01:0.5, lora:1Miriam2:0.4, lora:StokeRealV1:0.6|negative_prompt=(big head, huge head:1.2), (bangs,:1.3) head out of frame, out of canvas, nipples, (hands:1.3), long neck, big head, asian, chinese, cartoon, painting, illustration, (worst quality, low quality, normal quality:2), (extra limbs:1.3)|prompt_styles=[]|steps=23|sampler_index=11|latent_index=11|full_quality=True|restore_faces=False|tiling=False|n_iter=1|batch_size=1|cfg_scale=5.8|clip_skip=1|seed=-1.0|subseed=-1.0|subseed_strength=0|seed_resize_from_h=0|seed_resize_from_w=0||height=800|width=640|enable_hr=True|denoising_strength=0.35|hr_scale=2|hr_upscaler=ESRGAN 4x_NMKD-Superscale-SP_178000_G|hr_force=True|hr_second_pass_steps=14|hr_resize_x=0|hr_resize_y=0|image_cfg_scale=6|diffusers_guidance_rescale=0.7|refiner_steps=5|refiner_start=0.8|refiner_prompt=|refiner_negative=|override_settings_texts=[] First - slow run: 17:48:16 I can get many fast runs in a row...or slow runs in a row, sometimes I get a slow run after a single fast ran....I can't understand why, probably it's something small that I just don't understand - not a huge problem, just something that didn't happened with the previous version. *Edited so it's easier to see when the slow or fast run start |
Beta Was this translation helpful? Give feedback.
-
|
Sorry, by the "older version" I mean the version previous to the recent Major Update. I still have it installed in another folder, I just have to move a model/loras back to it if you want me to test it. I only have this problem since I did the clean install for the latest major update, never happened before. With the previous SD.Next version my generations took around ~35-40 seconds with the same settings as in the test above. Actually I used an image that I created with the previous version of SD.Next and I transfered the prompts and settings from the "Process image" to text just to be sure my weird problem is not caused by my img settings. Hope that make sense. |
Beta Was this translation helpful? Give feedback.
-
|
well, you're using both freeu and hypertile now and those were not present before, so its not really apples-to-apples compare. |
Beta Was this translation helpful? Give feedback.
-
|
Sorry for bothering you again but I found a way to "fix" it and it might be helpful for others with older GPUs or maybe you understand this better than I do (I'm sure you do) and find a way to properly resolve it....if it's actually worth the time. I "fixed" it by putting "Live preview display period" to -1. Now all generations are "fast runs" with one exception in 20 runs when it still showed me a preview frame but it was the only time it was a "slow run". I say "fixed" with quotation marks because I'm not sure if this was actually the problem or if it's just giving my GPU enough headroom to not have the garbage collector triggered during the process.
Is there a proper way to disable live previews entirely? Other things I did and I tested it after every single one without success - but it probably worth to mention them since they might affect the result I have....but I doubt it:
Probably a small bug/issue - Notification sound that play when the image is generated is often delayed and sometimes play twice, once the image is done upscaling or right before the upscaling start and once after ADetailer and final image is generated. PS: while writing this I keep generating images (one by one) on my other monitor, every once in a while it still show me a preview image frame and then it's a slow run...but it's so rare that I still consider this a "fix" |
Beta Was this translation helpful? Give feedback.
-
|
setting you may not like the answer, but i would never run system that close to limit. python and torch gc is non-deterministic, its a best-effort. so sometimes its going to clear up memory, sometimes it wont and you'll have issues. goal is to stay below 90%. but if you push it to the limit, this is what happens. btw, the fact that live preview consumes memory is normal and unavoidable - that's why live preview is disabled by default if you run with |
Beta Was this translation helpful? Give feedback.
-
|
Sadly even with Progress update period set to 0 it's still showing live previews from time to time (I asume Progress update period is show_progress_every_n_steps) and when it show live preview, it's 2x slower. Not tested properly but it looks like it even show live previews more often with 0 instead of 5000. It's not about liking or disliking your answer, I appreciate your answers and your work a lot...i'm just a bit frustrated that it never happened before and I'm trying to see if there is a way to make it work like it previously did. I had no problem generating with live preview on approximate before the update even if there was no hypertile and I used token merging. Token merging was ~10s slower but always consistent, without 2x slow runs on 640x800, now I have slow runs even on 512x768 (both cases with 2x upscale). But if there is no way, that is ok.
|
Beta Was this translation helpful? Give feedback.
-
|
i've made changes to client-side so browser never even asks for preview if previews are disabled on server. its in dev branch, will be merged to master next week. |
Beta Was this translation helpful? Give feedback.
-
|
Thank you for being patient with me and replying in terms that I can understand. You're doing a great job with everything! |
Beta Was this translation helpful? Give feedback.
i've made changes to client-side so browser never even asks for preview if previews are disabled on server. its in dev branch, will be merged to master next week.