Omost：自动帮你完善提示词并生成符合你预期的图像

Omost 是一个使用大语言模型（LLM）生成图像的项目。它通过虚拟画布代理来合成图像，旨在将模型的代码能力转化为图像生成能力。

具体来说，它通过使用语言模型编写代码，描述图像的各个部分及其布局，然后利用这些描述生成实际的图像。

• Omost 使用经过训练的大型语言模型（如 Llama3 和 Phi3）来生成描述图像的代码。这些模型可以理解并生成自然语言描述和编程代码。
• 模型生成的代码不是直接的图像，而是对图像内容的详细描述。这些描述包括图像中的各个元素（如人物、物体、背景等）的特征、位置和样式。

  Omost 的模型通过生成图像描述代码来实现图像合成。这些描述代码包括全局描述和局部描述：

  • 全局描述：用于描述整个图像的整体场景和氛围。
  • 局部描述：用于描述图像中各个部分的细节和位置。

  例如，生成一幅勇士与龙战斗的图像时，模型会生成描述勇士、龙、战场和背景天空等部分的代码。这些代码包含了各部分的位置、样式、细节描述和相对深度等信息。

• Omost 提供一个虚拟画布代理，用于接收和理解这些描述代码。画布代理根据代码将图像的各个部分组合起来，生成一个完整的图像描述。
• 虚拟画布代理将这些描述传递给具体的图像生成器。图像生成器根据描述生成最终的图像。生成器可以是任何能够理解这些描述并生成图像的工具或模型。

通俗解释Omost 是怎么工作的？

1. 文字描述： 你首先用文字描述你想要的图像内容。例如，你可能会描述：“一群勇士在与一条龙战斗，背景是闪电和暴风雨的天空。”

2. 生成代码： Omost 利用大语言模型会把你的文字描述转换成一种“代码”。这个代码不是普通的计算机代码，而是一种详细的描述，告诉电脑如何在画布上绘制出你想要的图像。

3. 虚拟画布： 这个代码会传递给一个虚拟画布。虚拟画布就像是一个画图的软件，它会根据代码里的描述来安排图像的各个部分。

4. 图像生成器： 最后，虚拟画布把这些安排传递给一个图像生成器，图像生成器会根据这些安排绘制出最终的图像。

举个例子

假设你想要一幅“勇士与龙战斗”的图像：

1. 你告诉 Omost：“我想要一幅勇士与龙战斗的图像，背景是闪电和暴风雨的天空。”
2. Omost 的模型会生成代码，详细描述勇士的位置、龙的样子、背景的天空等等。
3. 虚拟画布根据这些描述安排图像的各个部分。
4. 图像生成器根据这些安排绘制出一幅完整的图像。

场景：生成勇士与龙战斗的图像

用户和助手之间的对话示例展示了如何一步一步生成一个复杂场景的图像。具体步骤如下：

1、用户请求生成图像： 用户请求生成一个“勇士与龙战斗”的图像。

用户：
生成一个勇士与龙激烈战斗的图像

2、助手生成全局描述： 助手首先初始化一个画布，并设置整个图像的全局描述，描述了场景的总体情况和氛围。

# 初始化画布
canvas = Canvas()
# 设置全局描述
canvas.set_global_description(
description=’A fierce battle between warriors and a dragon.’,
detailed_descriptions=[
‘In this intense scene, a group of fierce warriors is engaged in an epic battle with a mighty dragon.’,
‘The warriors, clad in armor and wielding swords and shields, are positioned on the left side of the image.’,
‘The dragon, with its massive wings spread wide and its fiery breath illuminating the scene, dominates the center of the image.’,
‘The background is a dark, stormy sky with flashes of lightning, adding to the drama and tension of the battle.’
],
tags=’battle, warriors, dragon, fierce, armor, swords, shields, stormy sky, lightning, intense’,
HTML_web_color_name=’darkslategray’
)

3、助手生成局部描述： 助手接下来为图像中的各个部分添加详细描述。例如，描述勇士和龙的具体细节和位置。

# 添加勇士的描述
canvas.add_local_description(
location=’on the left’,
offset=’no offset’,
area=’a large horizontal area’,
distance_to_viewer=5.0,
description=’A group of fierce warriors.’,
detailed_descriptions=[
‘The warriors, clad in gleaming armor, are positioned on the left side of the image.’,
‘They are armed with swords, shields, and spears, ready for battle.’,
‘Their faces are set with determination and focus, reflecting their resolve to defeat the dragon.’
],
tags=’warriors, armor, swords, shields, determined, focused, battle’,
atmosphere=’Determined and focused, ready for the fierce battle.’,
style=’Highly detailed and dynamic, capturing the intensity of the warriors.’,
quality_meta=’High resolution with intricate details and dynamic poses.’,
HTML_web_color_name=’darkgoldenrod’
)

# 添加龙的描述
canvas.add_local_description(
location=’in the center’,
offset=’no offset’,
area=’a large square area’,
distance_to_viewer=7.0,
description=’A mighty dragon.’,
detailed_descriptions=[
‘The dragon is a massive creature, dominating the center of the image with its wide-spread wings and fiery breath.’,
‘Its scales glisten with a metallic sheen, reflecting the light from its fiery breath.’,
“The dragon’s eyes burn with a fierce intensity, and its teeth are sharp and menacing.”,
‘The wings of the dragon are powerful and spread wide, casting shadows over the battlefield.’,
‘The dragon’s body is muscular and covered in protective scales, with a long, sinewy tail that adds to its formidable appearance.’,
‘The fiery breath of the dragon illuminates the scene, casting a reddish glow over the warriors and the battlefield.’
],
tags=’dragon, massive, wings, fiery breath, glistening scales, metallic sheen, fierce eyes, sharp teeth, powerful wings, shadows, battlefield, muscular body, protective scales, sinewy tail, formidable, reddish glow, illumination’,
atmosphere=’Intense and menacing, with a powerful presence.’,
style=’Epic and dramatic, emphasizing the grandeur and danger of the dragon.’,
quality_meta=’High resolution with dramatic lighting and detailed textures.’,
HTML_web_color_name=’firebrick’
)

4、背景描述： 助手为图像的背景添加描述，如风暴天空和散落的战场。

# 添加风暴天空的描述
canvas.add_local_description(
location=’on the top’,
offset=’no offset’,
area=’a large horizontal area’,
distance_to_viewer=10.0,
description=’A stormy sky with flashes of lightning.’,
detailed_descriptions=[
‘The background of the image is a dark, stormy sky filled with swirling clouds and flashes of lightning.’,
‘The sky is turbulent, with clouds dark and foreboding, adding to the dramatic tension of the battle.’,
‘The lightning flashes illuminate the scene, casting sharp, brief lights over the warriors and the dragon.’,
‘The stormy sky creates a sense of chaos and unpredictability, heightening the intensity of the battle below.’,
‘The overall atmosphere is one of impending doom and relentless conflict, with the storm mirroring the fierce struggle between the warriors and the dragon.’
],
tags=’stormy sky, dark clouds, lightning, turbulent, foreboding, dramatic tension, illumination, chaos, unpredictability, intensity, impending doom, relentless conflict, battle, warriors, dragon, swirling clouds, sharp lights, brief lights’,
atmosphere=’Chaotic and intense, mirroring the fierce battle below.’,
style=’Dramatic and turbulent, emphasizing the conflict and tension.’,
quality_meta=’High resolution with dynamic lighting and detailed cloud textures.’,
HTML_web_color_name=’midnightblue’
)

# 添加战场描述
canvas.add_local_description(
location=’on the bottom’,
offset=’no offset’,
area=’a large horizontal area’,
distance_to_viewer=5.0,
description=’A debris-covered battlefield.’,
detailed_descriptions=[
‘The ground of the battlefield is littered with debris, remnants of previous battles.’,
‘Broken weapons, shattered shields, and scattered armor pieces are strewn across the battlefield.’,
‘The terrain is uneven, with patches of mud and dirt, adding to the sense of a long and brutal conflict.’,
‘The debris-covered battlefield enhances the chaotic and intense atmosphere of the scene, reflecting the ferocity and duration of the battle.’,
‘The overall appearance is one of destruction and turmoil, with the remnants of previous battles serving as a grim reminder of the ongoing struggle.’
],
tags=’battlefield, debris, broken weapons, shattered shields, scattered armor, uneven terrain, mud, dirt, brutal conflict, chaos, intensity, destruction, turmoil, previous battles, ongoing struggle, remnants, ferocity, duration’,
atmosphere=’Chaotic and intense, reflecting the ferocity of the battle.’,
style=’Detailed and realistic, emphasizing the destruction and chaos.’,
quality_meta=’High resolution with detailed textures and realistic debris.’,
HTML_web_color_name=’darkolivegreen’
)

5、对话编辑： 用户可以继续与助手对话，进行进一步的图像编辑。例如，用户要求将龙改为恐龙。

用户：
将龙改为恐龙

助手会更新图像描述：

# 修改全局描述中的龙为恐龙
canvas.set_global_description(
description=’A fierce battle between warriors and a dinosaur.’,
detailed_descriptions=[
‘In this intense scene, a group of fierce warriors is engaged in an epic battle with a mighty dinosaur.’,
‘The warriors, clad in armor and wielding swords and shields, are positioned on the left side of the image.’,
‘Their expressions are determined and focused, reflecting their resolve to defeat the dinosaur.’,
‘The dinosaur, with its massive body and powerful legs, dominates the center of the image.’,
‘Its scales glisten with a metallic sheen, and its eyes burn with a fierce intensity.’,
‘The background is a dark, stormy sky with flashes of lightning, adding to the drama and tension of the battle.’,
‘The ground is littered with debris and the remnants of previous battles, enhancing the sense of a long and brutal conflict.’,
‘The overall atmosphere is one of chaos and intensity, with the warriors and the dinosaur locked in a fierce struggle for survival.’
],
tags=’battle, warriors, dinosaur, fierce, armor, swords, shields, stormy sky, lightning, intense’,
HTML_web_color_name=’darkslategray’
)