Colab version

[bjornlarssen]

Here's my notebook, based on Andrew Wong's/Stable Diffusion Art one:

https://colab.research.google.com/drive/1N0GB9O3Q3EtBwFK1rzNBZkuiF6MUoLlE?usp=sharing

How to make it work:
I use Andrew's simplified folder structure, adding an extra folder for VladCfg. (See attachment.) It's quite self-explanatory. The Models folder copies into models, Lora into loras… etc. VladCfg is for .json/params.txt files. The repo gets cloned to Colab storage and doesn't use Google Drive space, and the models/LoRAs/etc. are sym-linked.

In the settings I had to change the folder structure when it came to outputs, but it's as simple as adding /content/drive/MyDrive/AI_PICS (in my case). Works.

What doesn't work:
• I can't disable extensions, which is apparently because I use --share, but if I don't, I can't connect due to me not being Google.
• SDXL images come out black. Always. VAE, no VAE, the --no-half-vae argument AND setting, black they are.
• I install xformers and then it uninstalls itself.
• The only way I found to add my extensions was to add them into the code. Andrew's original notebook has an "urls parser," but because I seriously have no clue what I am doing and Vlad's version goes "I don't know what urls are, bye," I just pasted the links.

The A1111 notebook by Andrew just stopped working – I think Google updated something, helpfully, so I either have to restart every 15 minutes, or don't even get to load the SDXL model at all. Technically, Vladmantic works better, at least with memory management (finally no crazy RAM leaks!!!) except black SDXL images are not exactly what I desire. Doesn't matter what I do with VAE either. The 1.5 models work.

If anybody can help me with the black SDXL images – and, ideally, help me disable/add extensions despite the --share argument – I will be the MOST grateful.

Also, I have no clue how the refiner works here, but that's unrelated to the notebook… On A1111 I use the refiner extension, which just runs the renders past the refiner, but since my results are black images anyway, I didn't see the point.

PS. Have I mentioned I have no clue what I am doing? Is this Python? Linux? What am I doing? Am I doing it twice? No clue. But it works, while A1111, right now, just doesn't.

Edit: The web UI tells me the render is finished, but it isn't – A1111 tends to be one minute behind, Vladmantic tends to be one minute in the future. I have to check the Colab notebook to see how it's actually progressing.

[bjornlarssen]

Update:

I haven't published the new version yet, because there are some Bits Of Shame (I have no clue what I am doing). Almost everything works, though. Even got SDXL to work! (That was the hardest part.)

My notebook installs SD and Vladmantic on the Colab drive, using symlinks to connect with models, LoRAs, etc. which are on my Google Drive. Using ngrok, I connect straight to localhost (i.e. Colab itself), which means I can skip the --share flag and change certain options I couldn't earlier. I haven't been able to compare, since A1111 is currently unusable on Colab, but it feels faster. I can't recreate my old images, though (probably because I switched to diffusers in order to use SDXL, and prompting doesn't work exactly like it does in A1111).

"Restart server" in A1111 resulted in having to restart the whole thing whether I wanted to or not, here you have to watch what Colab is doing, then just refresh the ngrok tab after 30 seconds or so (otherwise the "shutdown in progress" screen will stay there indefinitely). Watching what Colab is doing is unfortunately necessary most of the time, because the progress bar AND even displaying the results does not do what it should. Sometimes I think it crashed, or I get the previous images displayed, while it's still working.

In order to use some, but not all, 1.5 based models, I have to both load the other VAE and switch back from diffusers to original backend. The nice thing is that it doesn't require a restart. Sometimes it throws out random errors, and then I stop-restart the cell. Takes about 30 seconds.

I might be making this up, but SDXL seems to respond to my prompts much better than A1111 does/did. Which unfortunately also means I can't recreate my old images. (But I prefer the new ones ;) )

Here's my list of technical problems. Some of them may be unrelated to Colab.

• The RAM leak seems to have (re-)appeared with the new updated branch, but it's nowhere near as bad as A1111 (which is currently unusable). I would have to add a hash to pull the previous branch, because of course Colab requires me to install anew every time, and I just do !git clone.
• I can switch extensions off and on, but to install new ones manually I have to add !git clone in the code (otherwise I would have to symlink extensions to my Google Drive, too, and use even more space) – the notebook I based it on uses code I don't understand. I don't even know what I am working with when changing Colab code. Linux? Python? C++? Mystery Inka language to program pyramids? I'm that monkey with a typewriter, only my typewriter is really fast.
• I used a refiner extension before – https://github.com/wcde/sd-webui-refiner – on A1111, and it does all the refiner what-is-this-i-don't-even work for me. When I add it here, it simply doesn't appear anywhere but on the list of installed extensions.
• Therefore, I don't seem to understand how to use the refiner (I don't think that's Colab's fault) – using second pass with SDXL/diffusers seems to break the VAE (I am using the fix version), as if the VAE doesn't get applied after I use second pass.

The other problems are probably caused by me never having used SD Next before and being confused:

• in img2img the refiner won't load as a model, so I definitely don't know how to use it.
• I really, really wish there was an extension/option like A1111's PNG Info, which reads the metadata, then sends it to txt2img/img2img. I tried to install one, but it didn't appear anywhere, possibly because it's no longer supported. For now I use a metadata online extractor, but it's nowhere near as handy.

Once I figure out how to install extensions using a text field with URLs separated with commas, pass the arguments from a text field without python going "I don't know the argument {arg}" and figure out how to use the refiner without breaking the VAE, I'll make the notebook public. (If someone can tell me how to add the refiner extension I linked to above – or how to make the refiner work otherwise – I'll be very grateful.)

[djwashout]

Man. hats of to you. Trying to make all this work whin limited coding experience is hard. I haven't used any commands since dos 5.0 when win.exe was an app. lol. but here I am trying to learn python and writing notebooks. What about buying a google drive storage account, and a colab pro account and trying to install on gdrive? I run everything on my gdrive so i can have access to the files. I'm not a fan of the notebooks that use colab storage as i cant access the install files and mod anything.

[bjornlarssen]

I'm very impressed by myself ;)

I have Google Drive account and I use Colab Pro. (I don't know whether Vlad's notebook runs without paying, Google are little butches for making it purposefully impossible to run A1111 without paying…) So, I keep the files that I need specifically on the Drive, the install happens on the Colab storage, everything is symlinked. I only added the "VladCfg" folder because I am still hoping the A1111 notebook I was using gets fixed, but I think it's actually Google having updated something – tonight Vlad's notebook is also having that massive RAM leak which yesterday was noticeable, today it's restart-every-half-an-hour (but the A1111 was "actually I can't load the SDXL model at all").

What gets installed on Colab:
– code
– extensions

I think that's it. Everything else works via symlinks. I don't know why I am paying for Colab Pro+ and can't just access my own /content folder, but the symlinks work. Here's what I do:

models_in_google_drive = glob.glob(output_path + '/models/*') (something I don't understand – but it works so yay – that figures out what's in the folder on Google Drive)
print('Models in Google Drive: %s'%models_in_google_drive) (does what it says)
for f in models_in_google_drive: (for every model)
%cd {root}/stable-diffusion-webui/models/Stable-diffusion
!ln -s {f} (create a symlink that makes Colab think it's actually in its own folder when it's in my Drive)

[djwashout]

Here is a copy of a notebook I put together. You will need a colab pro account. It installs the A1111 repo on gdrive/mydrive.

https://colab.research.google.com/drive/18gCoukQOFkyX3Cy1uTF-tejuFzm9xeIC?usp=drive_link

[bjornlarssen]

THAT was the problem.

Colab was the problem.

Installing it on GDrive solved everything. I think they changed something with symlinks (the same notebooks, same settings, etc. suddenly going boink) so now that I removed the symlinks (I am using my "old" notebook, Andrew Wong's, with only one change – "save everything in Google Drive") the problems are gone.

Thank you so much! Also, thanks for your "updates," GOOGLE.

[djwashout]

Do you have links to other notebooks? Wouldn't be bad to keep a collection. And looking at others code helps me improve my skills.

Yea google ant dumb. lol. They are on to us installing in colab memory. You would think a company with more wealth than a nation wouldn't mind ppl running Ai on the platform, but nope! If they can get an extra dollar out of it they will. World domination! I'm actually kinda dreading AI taking over TBH.

[bjornlarssen]

I actually took Andrew's original notebook (which is paid, I'm afraid, but I was desperate to stop using TheLastBen's) and modified it to download everything to GD and retain a particular fork that works 100% of the time with 1.5 (but still has the RAM leak with SDXL because of the refiner…)

Why in the name of Melon would Google break code on their Colab drive? Isn't Colab for running code? Sigh. AI might turn out to be less greedy…