diff --git a/README.md b/README.md index 34a9f98..a16dda3 100644 --- a/README.md +++ b/README.md @@ -1,32 +1,50 @@ # Stable Diffusion for Remote Sensing Image Generation #### Author: Zhiqiang yuan @ AIR CAS, [Send a Email](yuan_zhi_qiang@sina.cn) +#### Welcome :+1:_`Fork and Star`_:+1:, then we'll let you know when we update +#### ------------------------------------------------------------------------------------- -A simple project for text-to-image remote sensing image generation, -and we will release the code of `using text to control regions for super-large RS image generation` later. +A simple project for `text-to-image remote sensing image generation`, +and we will release the code of `using multiple text to control regions for super-large RS image generation` later. Also welcome to see the project of [image-condition fake sample generation](https://github.com/xiaoyuan1996/Controllable-Fake-Sample-Generation-for-RS) in [TGRS, 2023](https://ieeexplore.ieee.org/abstract/document/10105619/). ## Environment configuration -Follow [original training repo](https://github.com/justinpinkney/stable-diffusion.git) . +Follow and thanks [original training repo](https://github.com/justinpinkney/stable-diffusion.git) . ## Pretrained weights -We used [RSITMD](https://github.com/xiaoyuan1996/AMFMN) as training data and fine-tuned stable diffusion for 10 epochs with 1 x A100 GPU. +We used RS image-text dataset [RSITMD](https://github.com/xiaoyuan1996/AMFMN) as training data and fine-tuned stable diffusion for 10 epochs with 1 x A100 GPU. When the batchsize is 4, the GPU memory consumption is about 40+ Gb during training, and about 20+ Gb during sampling. The pretrain weights is realesed at [last-pruned.ckpt](https://github.com/xiaoyuan1996/AMFMN). ## Using -Download the pretrain weights to current dir, and run with: + +### Samling +Download the pretrain weights `last-pruned.ckpt` to current dir, and run with: ```commandline -bash sample.sh +python scripts/txt2img.py \ + --prompt 'Some boats drived in the sea' \ + --outdir 'outputs/RS' \ + --H 512 --W 512 \ + --n_samples 4 \ + --config 'configs/stable-diffusion/RSITMD.yaml' \ + --ckpt './last-pruned.ckpt' ``` + +### Traing We will update the train code ASAP. ## Examples -**Caption:** Some boats drived in the sea. +**Caption:** `Some boats drived in the sea.` ![./assets/shows1.png](./assets/shows1.png) -**Caption:** A lot of cars parked in the airport. -![./assets/shows1.png](./assets/shows2.png) +**Caption:** `A lot of cars parked in the airport.` +![./assets/shows2.png](./assets/shows2.png) + +**Caption:** `A large number of vehicles are parked in the parking lot, next to the bare desert.` +![./assets/shows3.png](./assets/shows3.png) + +**Caption:** `There is a church in a dark green forest with two table tennis courts next to it.` +![./assets/shows4.png](./assets/shows4.png) diff --git a/assets/shows3.png b/assets/shows3.png new file mode 100644 index 0000000..b48012d Binary files /dev/null and b/assets/shows3.png differ diff --git a/assets/shows4.png b/assets/shows4.png new file mode 100644 index 0000000..8676525 Binary files /dev/null and b/assets/shows4.png differ diff --git a/sample.sh b/sample.sh deleted file mode 100644 index 708e6ae..0000000 --- a/sample.sh +++ /dev/null @@ -1,7 +0,0 @@ -python scripts/txt2img.py \ - --prompt 'There is a forest in the middle of the ocean' \ - --outdir 'outputs/RS' \ - --H 512 --W 512 \ - --n_samples 4 \ - --config 'configs/stable-diffusion/RSITMD.yaml' \ - --ckpt './last-pruned.ckpt' \ No newline at end of file