Narsil HF staff ysharma HF staff commited on
Commit
93db298
0 Parent(s):

Duplicate from ysharma/InstructPix2Pix_Chatbot

Browse files

Co-authored-by: yuvraj sharma <[email protected]>

Files changed (4) hide show
  1. .gitattributes +34 -0
  2. README.md +14 -0
  3. app.py +112 -0
  4. requirements.txt +4 -0
.gitattributes ADDED
@@ -0,0 +1,34 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ *.7z filter=lfs diff=lfs merge=lfs -text
2
+ *.arrow filter=lfs diff=lfs merge=lfs -text
3
+ *.bin filter=lfs diff=lfs merge=lfs -text
4
+ *.bz2 filter=lfs diff=lfs merge=lfs -text
5
+ *.ckpt filter=lfs diff=lfs merge=lfs -text
6
+ *.ftz filter=lfs diff=lfs merge=lfs -text
7
+ *.gz filter=lfs diff=lfs merge=lfs -text
8
+ *.h5 filter=lfs diff=lfs merge=lfs -text
9
+ *.joblib filter=lfs diff=lfs merge=lfs -text
10
+ *.lfs.* filter=lfs diff=lfs merge=lfs -text
11
+ *.mlmodel filter=lfs diff=lfs merge=lfs -text
12
+ *.model filter=lfs diff=lfs merge=lfs -text
13
+ *.msgpack filter=lfs diff=lfs merge=lfs -text
14
+ *.npy filter=lfs diff=lfs merge=lfs -text
15
+ *.npz filter=lfs diff=lfs merge=lfs -text
16
+ *.onnx filter=lfs diff=lfs merge=lfs -text
17
+ *.ot filter=lfs diff=lfs merge=lfs -text
18
+ *.parquet filter=lfs diff=lfs merge=lfs -text
19
+ *.pb filter=lfs diff=lfs merge=lfs -text
20
+ *.pickle filter=lfs diff=lfs merge=lfs -text
21
+ *.pkl filter=lfs diff=lfs merge=lfs -text
22
+ *.pt filter=lfs diff=lfs merge=lfs -text
23
+ *.pth filter=lfs diff=lfs merge=lfs -text
24
+ *.rar filter=lfs diff=lfs merge=lfs -text
25
+ *.safetensors filter=lfs diff=lfs merge=lfs -text
26
+ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
27
+ *.tar.* filter=lfs diff=lfs merge=lfs -text
28
+ *.tflite filter=lfs diff=lfs merge=lfs -text
29
+ *.tgz filter=lfs diff=lfs merge=lfs -text
30
+ *.wasm filter=lfs diff=lfs merge=lfs -text
31
+ *.xz filter=lfs diff=lfs merge=lfs -text
32
+ *.zip filter=lfs diff=lfs merge=lfs -text
33
+ *.zst filter=lfs diff=lfs merge=lfs -text
34
+ *tfevents* filter=lfs diff=lfs merge=lfs -text
README.md ADDED
@@ -0,0 +1,14 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ title: InstructPix2Pix Chatbot
3
+ emoji: ⚡
4
+ colorFrom: blue
5
+ colorTo: purple
6
+ sdk: gradio
7
+ sdk_version: 3.16.2
8
+ app_file: app.py
9
+ pinned: false
10
+ license: mit
11
+ duplicated_from: ysharma/InstructPix2Pix_Chatbot
12
+ ---
13
+
14
+ Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
app.py ADDED
@@ -0,0 +1,112 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import PIL
2
+ import requests
3
+ import torch
4
+ import gradio as gr
5
+ import random
6
+ from PIL import Image
7
+ import os
8
+ import time
9
+ from diffusers import StableDiffusionInstructPix2PixPipeline, EulerAncestralDiscreteScheduler
10
+
11
+ #Loading from Diffusers Library
12
+ model_id = "timbrooks/instruct-pix2pix"
13
+ pipe = StableDiffusionInstructPix2PixPipeline.from_pretrained(model_id, torch_dtype=torch.float16, revision="fp16") #, safety_checker=None)
14
+ pipe.to("cuda")
15
+ pipe.enable_attention_slicing()
16
+
17
+ counter = 0
18
+
19
+
20
+ help_text = """ Note: Functionality to revert your changes to previous/original image can be released in future versions. For now only forward editing is available.
21
+
22
+ Some notes from the official [instruct-pix2pix](https://huggingface.co/spaces/timbrooks/instruct-pix2pix) Space by the authors
23
+ and from the official [Diffusers docs](https://huggingface.co/docs/diffusers/main/en/api/pipelines/stable_diffusion/pix2pix) -
24
+
25
+ If you're not getting what you want, there may be a few reasons:
26
+ 1. Is the image not changing enough? Your guidance_scale may be too low. It should be >1. Higher guidance scale encourages to generate images
27
+ that are closely linked to the text `prompt`, usually at the expense of lower image quality. This value dictates how similar the output should
28
+ be to the input. This pipeline requires a value of at least `1`. It's possible your edit requires larger changes from the original image.
29
+
30
+ 2. Alternatively, you can toggle image_guidance_scale. Image guidance scale is to push the generated image towards the inital image. Image guidance
31
+ scale is enabled by setting `image_guidance_scale > 1`. Higher image guidance scale encourages to generate images that are closely
32
+ linked to the source image `image`, usually at the expense of lower image quality.
33
+
34
+ 3. I have observed that rephrasing the instruction sometimes improves results (e.g., "turn him into a dog" vs. "make him a dog" vs. "as a dog").
35
+
36
+ 4. Increasing the number of steps sometimes improves results.
37
+
38
+ 5. Do faces look weird? The Stable Diffusion autoencoder has a hard time with faces that are small in the image. Try:
39
+ * Cropping the image so the face takes up a larger portion of the frame.
40
+ """
41
+
42
+ def chat(image_in, in_steps, in_guidance_scale, in_img_guidance_scale, image_hid, img_name, counter_out, prompt, history, progress=gr.Progress(track_tqdm=True)):
43
+ progress(0, desc="Starting...")
44
+ #if message == "revert": --to add revert functionality later
45
+ if counter_out > 0:
46
+ edited_image = pipe(prompt, image=image_hid, num_inference_steps=int(in_steps), guidance_scale=float(in_guidance_scale), image_guidance_scale=float(in_img_guidance_scale)).images[0]
47
+ if os.path.exists(img_name):
48
+ os.remove(img_name)
49
+ temp_img_name = img_name[:-4]+str(int(time.time()))+'.png'
50
+ # Create a file-like object
51
+ with open(temp_img_name, "wb") as fp:
52
+ # Save the image to the file-like object
53
+ edited_image.save(fp)
54
+ #Get the name of the saved image
55
+ saved_image_name = fp.name
56
+ #edited_image.save(temp_img_name) #, overwrite=True)
57
+ counter_out += 1
58
+ else:
59
+ seed = random.randint(0, 1000000)
60
+ img_name = f"./edited_image_{seed}.png"
61
+ edited_image = pipe(prompt, image=image_in, num_inference_steps=int(in_steps), guidance_scale=float(in_guidance_scale), image_guidance_scale=float(in_img_guidance_scale)).images[0]
62
+ if os.path.exists(img_name):
63
+ os.remove(img_name)
64
+ with open(img_name, "wb") as fp:
65
+ # Save the image to the file-like object
66
+ edited_image.save(fp)
67
+ #Get the name of the saved image
68
+ saved_image_name2 = fp.name
69
+ history = history or []
70
+ #Resizing (or not) the image for better display and adding supportive sample text
71
+ add_text_list = ["There you go", "Enjoy your image!", "Nice work! Wonder what you gonna do next!", "Way to go!", "Does this work for you?", "Something like this?"]
72
+ if counter_out > 0:
73
+ response = random.choice(add_text_list) + '<img src="/file=' + saved_image_name + '">'
74
+ history.append((prompt, response))
75
+ return history, history, edited_image, temp_img_name, counter_out
76
+ else:
77
+ response = random.choice(add_text_list) + '<img src="/file=' + saved_image_name2 + '">' #IMG_NAME
78
+ history.append((prompt, response))
79
+ counter_out += 1
80
+ return history, history, edited_image, img_name, counter_out
81
+
82
+
83
+ with gr.Blocks() as demo:
84
+ gr.Markdown("""<h1><center> Chat Interface with InstructPix2Pix: Give Image Editing Instructions</h1></center>
85
+ <p>For faster inference without waiting in the queue, you may duplicate the space and upgrade to GPU in settings.<br/>
86
+ <a href="https://huggingface.co/spaces/ysharma/InstructPix2Pix_Chatbot?duplicate=true">
87
+ <img style="margin-top: 0em; margin-bottom: 0em" src="https://bit.ly/3gLdBN6" alt="Duplicate Space"></a>
88
+ **Note: Please be advised that a safety checker has been implemented in this public space.
89
+ Any attempts to generate inappropriate or NSFW images will result in the display of a black screen
90
+ as a precautionary measure for the protection of all users. We appreciate your cooperation in
91
+ maintaining a safe and appropriate environment for all members of our community.**
92
+ <p/>""")
93
+ with gr.Row():
94
+ with gr.Column():
95
+ image_in = gr.Image(type='pil', label="Original Image")
96
+ text_in = gr.Textbox()
97
+ state_in = gr.State()
98
+ b1 = gr.Button('Edit the image!')
99
+ with gr.Accordion("Advance settings for Training and Inference", open=False):
100
+ gr.Markdown("Advance settings for - Number of Inference steps, Guidanace scale, and Image guidance scale.")
101
+ in_steps = gr.Number(label="Enter the number of Inference steps", value = 20)
102
+ in_guidance_scale = gr.Slider(1,10, step=0.5, label="Set Guidance scale", value=7.5)
103
+ in_img_guidance_scale = gr.Slider(1,10, step=0.5, label="Set Image Guidance scale", value=1.5)
104
+ image_hid = gr.Image(type='pil', visible=False)
105
+ img_name_temp_out = gr.Textbox(visible=False)
106
+ counter_out = gr.Number(visible=False, value=0, precision=0)
107
+ chatbot = gr.Chatbot()
108
+ b1.click(chat,[image_in, in_steps, in_guidance_scale, in_img_guidance_scale, image_hid, img_name_temp_out,counter_out, text_in, state_in], [chatbot, state_in, image_hid, img_name_temp_out, counter_out]) #, queue=True)
109
+ gr.Markdown(help_text)
110
+
111
+ demo.queue(concurrency_count=10)
112
+ demo.launch(debug=True, width="80%", height=2000)
requirements.txt ADDED
@@ -0,0 +1,4 @@
 
 
 
 
 
1
+ git+https://github.com/huggingface/diffusers.git
2
+ transformers
3
+ accelerate
4
+ safetensors