{"id":157,"date":"2023-02-20T10:46:43","date_gmt":"2023-02-20T09:46:43","guid":{"rendered":"https:\/\/project.inria.fr\/dare\/?page_id=157"},"modified":"2025-12-23T18:13:48","modified_gmt":"2025-12-23T17:13:48","slug":"sgc","status":"publish","type":"page","link":"https:\/\/project.inria.fr\/dare\/publications\/sgc\/","title":{"rendered":"SGC: Semantic based generative compression of images at extremely low bitrates"},"content":{"rendered":"\n<div class=\"entry-content\">\n<p><strong>Authors: <\/strong>Tom Bordin and Thomas Maugey<strong><br \/><\/strong><\/p>\n<p><strong>Abstract:<\/strong><\/p>\n<p><em>We propose a framework for image compression in which the fidelity criterion is replaced by semantic preservation objectives. Encoding the image thus becomes a simple extraction of semantic enabling to reach drastic compression ratio. The decoding side is handled by a generative model relying on the diffusion process for the reconstruction of images. We propose to describe the semantic using low resolution segmentation maps as guide. We further improve the generation introducing colors map guidance without retraining the generative decoder. We show that it is possible to produce images of high visual quality with preserved semantic at a bitrate competitive with classical codecs.<\/em><strong><br \/><\/strong><\/p>\n<p><strong>Contributions:<\/strong><\/p>\n<ul>\n<li>framework for image compression with a semantic representation<\/li>\n<li>image generation guided with colors on a trained Latent Diffusion Model<\/li>\n<li>visually competitive results with VVC at extremely low bitrates<\/li>\n<\/ul>\n<p><strong>Supplementary results:<\/strong><\/p>\n<p><a href=\"https:\/\/project.inria.fr\/dare\/publications\/sgc\/generated_examples0-2\/\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-177 size-full\" src=\"https:\/\/project.inria.fr\/dare\/files\/2023\/02\/Generated_examples0.png\" alt=\"\" width=\"3003\" height=\"1711\" srcset=\"https:\/\/project.inria.fr\/dare\/files\/2023\/02\/Generated_examples0.png 3003w, https:\/\/project.inria.fr\/dare\/files\/2023\/02\/Generated_examples0-300x171.png 300w, https:\/\/project.inria.fr\/dare\/files\/2023\/02\/Generated_examples0-1024x583.png 1024w, https:\/\/project.inria.fr\/dare\/files\/2023\/02\/Generated_examples0-768x438.png 768w, https:\/\/project.inria.fr\/dare\/files\/2023\/02\/Generated_examples0-1536x875.png 1536w, https:\/\/project.inria.fr\/dare\/files\/2023\/02\/Generated_examples0-2048x1167.png 2048w, https:\/\/project.inria.fr\/dare\/files\/2023\/02\/Generated_examples0-150x85.png 150w\" sizes=\"auto, (max-width: 3003px) 100vw, 3003px\" \/><\/a><\/p>\n<p><a href=\"https:\/\/project.inria.fr\/dare\/files\/2023\/02\/Generated_examples1.png\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-178\" src=\"https:\/\/project.inria.fr\/dare\/files\/2023\/02\/Generated_examples1.png\" alt=\"\" width=\"3003\" height=\"1672\" srcset=\"https:\/\/project.inria.fr\/dare\/files\/2023\/02\/Generated_examples1.png 3003w, https:\/\/project.inria.fr\/dare\/files\/2023\/02\/Generated_examples1-300x167.png 300w, https:\/\/project.inria.fr\/dare\/files\/2023\/02\/Generated_examples1-1024x570.png 1024w, https:\/\/project.inria.fr\/dare\/files\/2023\/02\/Generated_examples1-768x428.png 768w, https:\/\/project.inria.fr\/dare\/files\/2023\/02\/Generated_examples1-1536x855.png 1536w, https:\/\/project.inria.fr\/dare\/files\/2023\/02\/Generated_examples1-2048x1140.png 2048w, https:\/\/project.inria.fr\/dare\/files\/2023\/02\/Generated_examples1-150x84.png 150w\" sizes=\"auto, (max-width: 3003px) 100vw, 3003px\" \/><\/a><\/p>\n<p><a href=\"https:\/\/project.inria.fr\/dare\/files\/2023\/02\/Generated_examples2.png\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-181\" src=\"https:\/\/project.inria.fr\/dare\/files\/2023\/02\/Generated_examples2.png\" alt=\"\" width=\"3003\" height=\"1672\" srcset=\"https:\/\/project.inria.fr\/dare\/files\/2023\/02\/Generated_examples2.png 3003w, https:\/\/project.inria.fr\/dare\/files\/2023\/02\/Generated_examples2-300x167.png 300w, https:\/\/project.inria.fr\/dare\/files\/2023\/02\/Generated_examples2-1024x570.png 1024w, https:\/\/project.inria.fr\/dare\/files\/2023\/02\/Generated_examples2-768x428.png 768w, https:\/\/project.inria.fr\/dare\/files\/2023\/02\/Generated_examples2-1536x855.png 1536w, https:\/\/project.inria.fr\/dare\/files\/2023\/02\/Generated_examples2-2048x1140.png 2048w, https:\/\/project.inria.fr\/dare\/files\/2023\/02\/Generated_examples2-150x84.png 150w\" sizes=\"auto, (max-width: 3003px) 100vw, 3003px\" \/><\/a><\/p>\n<p><a href=\"https:\/\/project.inria.fr\/dare\/files\/2023\/02\/Generated_examples4.png\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-183\" src=\"https:\/\/project.inria.fr\/dare\/files\/2023\/02\/Generated_examples4.png\" alt=\"\" width=\"3003\" height=\"1672\" srcset=\"https:\/\/project.inria.fr\/dare\/files\/2023\/02\/Generated_examples4.png 3003w, https:\/\/project.inria.fr\/dare\/files\/2023\/02\/Generated_examples4-300x167.png 300w, https:\/\/project.inria.fr\/dare\/files\/2023\/02\/Generated_examples4-1024x570.png 1024w, https:\/\/project.inria.fr\/dare\/files\/2023\/02\/Generated_examples4-768x428.png 768w, https:\/\/project.inria.fr\/dare\/files\/2023\/02\/Generated_examples4-1536x855.png 1536w, https:\/\/project.inria.fr\/dare\/files\/2023\/02\/Generated_examples4-2048x1140.png 2048w, https:\/\/project.inria.fr\/dare\/files\/2023\/02\/Generated_examples4-150x84.png 150w\" sizes=\"auto, (max-width: 3003px) 100vw, 3003px\" \/><\/a><\/p>\n<p><a href=\"https:\/\/project.inria.fr\/dare\/files\/2023\/02\/Generated_examples3.png\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-182\" src=\"https:\/\/project.inria.fr\/dare\/files\/2023\/02\/Generated_examples3.png\" alt=\"\" width=\"3007\" height=\"1672\" srcset=\"https:\/\/project.inria.fr\/dare\/files\/2023\/02\/Generated_examples3.png 3007w, https:\/\/project.inria.fr\/dare\/files\/2023\/02\/Generated_examples3-300x167.png 300w, https:\/\/project.inria.fr\/dare\/files\/2023\/02\/Generated_examples3-1024x569.png 1024w, https:\/\/project.inria.fr\/dare\/files\/2023\/02\/Generated_examples3-768x427.png 768w, https:\/\/project.inria.fr\/dare\/files\/2023\/02\/Generated_examples3-1536x854.png 1536w, https:\/\/project.inria.fr\/dare\/files\/2023\/02\/Generated_examples3-2048x1139.png 2048w, https:\/\/project.inria.fr\/dare\/files\/2023\/02\/Generated_examples3-150x83.png 150w\" sizes=\"auto, (max-width: 3007px) 100vw, 3007px\" \/><\/a><\/p>\n<p><a href=\"https:\/\/project.inria.fr\/dare\/files\/2023\/02\/Generated_examples6.png\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-184\" src=\"https:\/\/project.inria.fr\/dare\/files\/2023\/02\/Generated_examples6.png\" alt=\"\" width=\"3006\" height=\"1672\" srcset=\"https:\/\/project.inria.fr\/dare\/files\/2023\/02\/Generated_examples6.png 3006w, https:\/\/project.inria.fr\/dare\/files\/2023\/02\/Generated_examples6-300x167.png 300w, https:\/\/project.inria.fr\/dare\/files\/2023\/02\/Generated_examples6-1024x570.png 1024w, https:\/\/project.inria.fr\/dare\/files\/2023\/02\/Generated_examples6-768x427.png 768w, https:\/\/project.inria.fr\/dare\/files\/2023\/02\/Generated_examples6-1536x854.png 1536w, https:\/\/project.inria.fr\/dare\/files\/2023\/02\/Generated_examples6-2048x1139.png 2048w, https:\/\/project.inria.fr\/dare\/files\/2023\/02\/Generated_examples6-150x83.png 150w\" sizes=\"auto, (max-width: 3006px) 100vw, 3006px\" \/><\/a><\/p>\n<p><a href=\"https:\/\/project.inria.fr\/dare\/files\/2023\/02\/Generated_examples7.png\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-185\" src=\"https:\/\/project.inria.fr\/dare\/files\/2023\/02\/Generated_examples7.png\" alt=\"\" width=\"3003\" height=\"1672\" srcset=\"https:\/\/project.inria.fr\/dare\/files\/2023\/02\/Generated_examples7.png 3003w, https:\/\/project.inria.fr\/dare\/files\/2023\/02\/Generated_examples7-300x167.png 300w, https:\/\/project.inria.fr\/dare\/files\/2023\/02\/Generated_examples7-1024x570.png 1024w, https:\/\/project.inria.fr\/dare\/files\/2023\/02\/Generated_examples7-768x428.png 768w, https:\/\/project.inria.fr\/dare\/files\/2023\/02\/Generated_examples7-1536x855.png 1536w, https:\/\/project.inria.fr\/dare\/files\/2023\/02\/Generated_examples7-2048x1140.png 2048w, https:\/\/project.inria.fr\/dare\/files\/2023\/02\/Generated_examples7-150x84.png 150w\" sizes=\"auto, (max-width: 3003px) 100vw, 3003px\" \/><\/a><\/p>\n<p><a href=\"https:\/\/project.inria.fr\/dare\/files\/2023\/02\/Generated_examples5-scaled.jpg\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-187\" src=\"https:\/\/project.inria.fr\/dare\/files\/2023\/02\/Generated_examples5-scaled.jpg\" alt=\"\" width=\"2560\" height=\"1426\" srcset=\"https:\/\/project.inria.fr\/dare\/files\/2023\/02\/Generated_examples5-scaled.jpg 2560w, https:\/\/project.inria.fr\/dare\/files\/2023\/02\/Generated_examples5-300x167.jpg 300w, https:\/\/project.inria.fr\/dare\/files\/2023\/02\/Generated_examples5-1024x570.jpg 1024w, https:\/\/project.inria.fr\/dare\/files\/2023\/02\/Generated_examples5-768x428.jpg 768w, https:\/\/project.inria.fr\/dare\/files\/2023\/02\/Generated_examples5-1536x856.jpg 1536w, https:\/\/project.inria.fr\/dare\/files\/2023\/02\/Generated_examples5-2048x1141.jpg 2048w, https:\/\/project.inria.fr\/dare\/files\/2023\/02\/Generated_examples5-150x84.jpg 150w\" sizes=\"auto, (max-width: 2560px) 100vw, 2560px\" \/><\/a><a href=\"https:\/\/project.inria.fr\/dare\/files\/2023\/02\/Generated_examples8.png\" data-wp-editing=\"1\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-186 size-thumbnail\" src=\"https:\/\/project.inria.fr\/dare\/files\/2023\/02\/Generated_examples8-150x150.png\" alt=\"\" width=\"150\" height=\"150\" \/><\/a><\/p>\n<\/div>\n","protected":false},"excerpt":{"rendered":"<p>Authors: Tom Bordin and Thomas Maugey Abstract: We propose a framework for image compression in which the fidelity criterion is replaced by semantic preservation objectives. Encoding the image thus becomes a simple extraction of semantic enabling to reach drastic compression ratio. The decoding side is handled by a generative model\u2026<\/p>\n<p> <a class=\"continue-reading-link\" href=\"https:\/\/project.inria.fr\/dare\/publications\/sgc\/\"><span>Continue reading<\/span><i class=\"crycon-right-dir\"><\/i><\/a> <\/p>\n","protected":false},"author":1433,"featured_media":0,"parent":119,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"class_list":["post-157","page","type-page","status-publish","hentry"],"_links":{"self":[{"href":"https:\/\/project.inria.fr\/dare\/wp-json\/wp\/v2\/pages\/157","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/project.inria.fr\/dare\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/project.inria.fr\/dare\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/project.inria.fr\/dare\/wp-json\/wp\/v2\/users\/1433"}],"replies":[{"embeddable":true,"href":"https:\/\/project.inria.fr\/dare\/wp-json\/wp\/v2\/comments?post=157"}],"version-history":[{"count":13,"href":"https:\/\/project.inria.fr\/dare\/wp-json\/wp\/v2\/pages\/157\/revisions"}],"predecessor-version":[{"id":325,"href":"https:\/\/project.inria.fr\/dare\/wp-json\/wp\/v2\/pages\/157\/revisions\/325"}],"up":[{"embeddable":true,"href":"https:\/\/project.inria.fr\/dare\/wp-json\/wp\/v2\/pages\/119"}],"wp:attachment":[{"href":"https:\/\/project.inria.fr\/dare\/wp-json\/wp\/v2\/media?parent=157"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}