Looking for an open source image-to-video to test with Intel ARC GPU

Hi,

I have A770 INTEL ARC card and I’m looking an open source AI API to generate some videos from an image.
A lot of thing I’ve found over the web is complicated (for me) to make it working with Intel ARC GPU. (ComfyUI works fine for me but just for generating images)

Could you give me some guidelines or AI open source solution or tools, to try please ?

I’ve read that INTEL AI Playground is great, but I don’t see any txt-to-video or img-to-video integrated tools

Best regards

PS : I have asked to INTEL support but they have suggested me to post my question here.

1 Like

After installing PyTorch for XPU, you should be able to generate it using Diffusers. It will probably take a long time…

.to(“cuda”) is for nVidia, so if you have an Intel GPU, replace it with .to(“xpu”).

#.to("cuda")
.to("xpu")

Thank you for your reply, I will look at it.

Meanwhile, I 've began to try Atomatic1111. could you tell me where we can I verify that the to(XPU) is well set please ?
(I have launched a working process but I can see my Intel GPU is not used)

1 Like

Try IPEX.

Sorry I’m a Beginner, I’m sure to understand what you mean by IPEX.

I’m using Automatic1111 with the InfiniteZoom, but the process it is very long : I think my Intel GPU is not used.

you meant using IPEX within my Automatic1111 configuration ?

1 Like

That’s about it.

I think even advanced users would have a hard time using A1111 WebUI with a GPU other than nVidia…
Intel ARC and RADEON themselves are for advanced users.
Also, if you use Extensions such as InfiniteZoom, it may be a little pointless if they are not compatible with Intel GPUs.

In addition to IPEX, there is also a method for accelerating Intel chips called OpenVINO, but this is difficult in its own way. Probably IPEX or DirectML would be better this time.

Intel® Extension for PyTorch (IPEX)

Command Line Arguments and Settings

--use-ipex

Any way to run this on Intel Arc? #455 (WebUI Forge)

sd-webui-ipex-enhancement

Thanks a lot for all your links.

If I really need to generate Infinite Zoom, which other framework (than A1111) would you recommend me fully compatible with Intel Arc ?

A1111+InfiniteZoom is the only easy way GUI I’ve found over the web…
(unfortunately, I did not find any other simple resource to do it easily)

1 Like

It looks like ComfyUI also has a plugin, and it seems to be compatible with Intel GPUs.
Also, although it’s not a GUI, I found something that looks like it’s for Intel’s OpenVINO.

However, basically, you should think that you will need a lot of extra effort and skill for anything you do other than CUDA…

https://docs.openvino.ai/2023.3/notebooks/236-stable-diffusion-v2-infinite-zoom-with-output.html

Really, ComfUI has a plugin for it ?
that’s a good news … So I have to go back with ComfyUI,

Few weeks ago, I succeeded in generating images with it, and it was full working with Intel GPU. But I never succeeded in getting it to work with OpenVino so i’ve forgot it.

But ok, I will try again with this plugin, thank you.

1 Like

Perhaps just similar one.
https://weirdwonderfulai.art/comfyui/infinite-zoom-loop-in-comfyui/

Hi,

I’ve reinstalled first ComfyUI that works fine to generate an image.
but when launching I can see some import errors:

WARNING: some comfy_extras/ nodes did not import correctly. This may be because they are missing some dependencies.
-IMPORT FAILED: nodes_canny.py
-IMPORT FAILED: nodes_morphology.py
-IMPORT FAILED: nodes_audio.py
-IMPORT FAILED: nodes_video.py
=> This issue might be caused by new missing dependencies added the last time you updated ComfyUI.
Please do a: pip install -r requirements.txt

Could you tell me how to correct the issue ?

I’m not using “requirements.txt” but “requirements-ipex.txt
And when installing ComfyUI the first I’ve already launched this command line:

pip install -r requirements-ipex.txt

But the issue is still there…

1 Like

I don’t know how to fix the error in ComfyUI…
I think the situation is probably the same as the github issue below, so please try changing the file and folder names to match your environment.

To be more specific, you can have as many Python installations as you like on the same PC, so there’s no point in installing the library in a Python other than the target one, and you definitely need to install the library in the Python that ComfyUI is using. If it still doesn’t work, it’s probably not compatible and won’t work.

Hi,
Thank you. I think I’ve previously followed the bad tutorial to install ComfyUI.

So I’ve installed it again to the letter, with the Intel ARC instructions below:

… and now all is fine. The Defaut worflow works fine and generate well images.

Now, I’m trying a simple thing but I have a very weird error :
if I replace the “Empty latent image” node by these 3 nodes : LoadImage+PadImageForInpainting+VAEencode (OR same thing with: LoadImage+VAEencode),

I obtain the Error:
KSampler : XPU out of memory. Tried to allocate 128.00 GiB. GPU 0 has a total capacity of 15.56 GiB. Of the allocated memory 2.50 GiB is allocated by PyTorch, and 39.26 MiB is reserved by PyTorch but unallocated. Please use empty_cache to release all unoccupied cached memory.

but if I let ComfyUI generates the image it works fine.

Would you have any idea of the cause ?

1 Like

I’ve followed this tutorial and all works fine until 6’00" :
Outpainting

but after 6’00’', when I replace VAEdecode+SaveImage by LoadImage+PadImageForInpainting+VAEencode, I obtain that big error…

Any idea ?

1 Like