[Catjourney #002] How to Combine Midjourney with 3D

Prompt: A hyperrealistic model with delicate, watercolor-inspired makeup, soft pastel washes on the eyelids and cheeks, against an ethereal, blurred floral background. The gentle, diffuse lighting enhances the watercolor effect, creating a dreamy, serene atmosphere. Created Using: DSLR camera, macro lens for detailed close-ups, watercolor makeup techniques, natural light with reflectors, high dynamic range imaging, natural style --ar 9:16 --style raw

🌟 English Version:

Recently, with the update of Framer's new components, our Catjourney encountered some issues that prevented us from publishing new content, so we had to revert the content. But don't worry, the newly uploaded content will be uploaded again later.

During this time, we contacted Framer's team members to troubleshoot the problem together. Although Framer reported errors, it did not specify where the errors were, and we eventually found that there were issues with modules we couldn't modify. It was a bit of a headache, but fortunately, we finally resolved it with the help of Sahil from Framer Team! A little moment of happiness 👇🌹 Thank U Sahil!

This week, I want to share with you some progress in 3D + AI. In fact, AI generating 3D is very similar to the early beta testing period of Midjourney. Many people did not have a foundation in drawing, but today we see the maturity of the MJ and SD ecosystem, allowing more and more people who were not originally in related industries to grow rapidly; similarly, for those who are already proficient in design tools, work efficiency has directly increased by two orders of magnitude.

Yes, besides AI, you also need to learn some 'editing' skills to maximize the benefits of design. If you want to seize the next opportunity, it's undoubtedly 3D. You know, the benefit of AI in the field of images is actually to reduce the 'learning cost' and 'work cost' (such as time and energy), encouraging more players to participate (not being intimidated by the difficulty of learning) and more user needs to be met and stimulated. Especially for those scene designs, which are time-consuming and labor-intensive...

For 3D, the learning cost is even greater. Even if you are a veteran, building detailed and usable scenes, characters, etc., takes more time. If you follow MR, AVP (Apple Vision Pro), game production, 3D software development, you will know that these things can sometimes be cooler than flat surfaces, as they can be interacted with, bringing a better and more immersive experience. In other words, we don't dislike 3D, but the threshold is too high, the cost is too great, and it's too difficult to enter.

What if the cost of 3D could be reduced by two orders of magnitude? On one level, it's like Midjourney and SD igniting flat content; at the same time, many new content platforms have emerged (you can see many AI content platforms). Just like the普及and progress of mobile phone cameras to Tiktok and Douyin.

Let's take a look at the latest exploration from @Yokohara , who used Tripo to generate a model from a 'single image' in 10 seconds, and then achieved the following amazing effect by mixing normal maps and videos 👇 You can take this opportunity to glimpse the huge power of 3D in the next year.

If you are a content creator, I strongly recommend that you start learning basic 3D editing skills from now on to prepare for the arrival of 3D; just as when Midjourney was quietly growing, I started learning various new design skills. Later, when I made B station tutorial videos, I didn't feel any production difficulties.

In summary, there will be many things we can do in the future. Agents + interactive 3D characters appear in front of you, what kind of experience would that be? You may know about the DAN model, but what if the characters are right beside you? You may want to learn scientific research, but what if the tutor is right in front of you... From this perspective, 3D is not art design, but it can prepare your development for the concept of 'spatial computing'.

(It's time for some fantasy 🤡...)

Next, I will share a simple workflow that I have done, combining Midjourney + Magnific + Tripo + Blender, suitable for art promotion. The learning cost is very low, but the effect is quite good. You know, new technologies are always tested and popularized within the circle 🔥, and it takes months or even years for the outside world to receive them. First, let's look at the effect 👇

Step one: First, use Midjourney to generate a mecha.

Then use Midjourney to generate different style texture maps as Magnific's Style Transfer Reference Images (style reference pictures). I generated 4 here, which are: fire, water, Mars, and camouflage styles. Then I pulled the Structure Strength to 100% to ensure the consistency of the lines.

Then you will get these 4 results.

Step two: It's very simple, just give the pictures as input directly to Tripo to generate 3D models, which you can try for free. I like to use the Arc browser to open multiple windows to improve efficiency, haha. Tripo, similar to Midjourney, will first generate preliminary results (Draft), and then you can Retry or Refine (similar to Upscale) for the models you like.

But currently, I prefer the drafts generated in the first stage because Tripo's Refine new algorithm has not been updated yet, while the new algorithm for drafts has been launched and the effect is great! (Also, remember to click the ❤️ heart symbol to put the draft model into the Favorite folder to prevent loss.)

On the other hand, this is also good because the draft model only takes a few seconds. In actual 3D art promotion work, it has already reduced 2/3 of the working time (from the feeling of a Blender Expert). This old wizard has a different approach from us because the project has higher requirements, and he will disassemble every part of the model very finely, then use AI to generate and reassemble (as well as various subsequent professional operations...).

Next, I downloaded the model and imported it into Blender, added a varnish-like material, added an HDRI scene, and lit it up, and it's OK. If you want to get a higher quality promotional picture, you can also use Magnific's Upscale to enlarge the picture.

Finally, add the LOGO and it's done. The advantage of 3D is that you can also bind bones to make animations and switch angles at will. If you want to use Video2Video, 3D as the input for the video will not cause line disorder, and the stability of large-scale movements is guaranteed. Here, stability includes edge lines and lighting in the scene. Video tutorial Click Here to view.

That's it for this issue's sharing! Catjourney will be back next week～

🌟 Chinese Version:

最近随着 Framer 新组件的更新，我们的 Catjourney 遇到了一些问题导致无法发布新的内容，所以我们不得已对内容进行了回退。不过别担心，新上传的内容稍后会重新上传一遍。

在这期间我们联络到了 Framer 的团队成员一起排查问题。因为 Framer 虽然报错，却不报具体错在哪，最后发现涉及到我们无法修改的模块有些问题。这令人有些头疼，幸运的是最终在 Sahil 的帮助下解决了！小确幸时刻 👇🌹 感谢他！

这周给大家分享一些 3D + AI 的进展。事实上，AI 生成 3D 非常类似于早期 Midjourney 内测的时间段。很多人并没有绘画基础，但今天我们看到 MJ 以及 SD 生态的成熟，让越来越多原本不是相关行业的从业者迅速成长起来；同样地，对于那些原本就熟练掌握设计工具的从业者来说，工作效率也直接飞跃 2 个数量级。

是的，除了 AI，你也必须学会一些「编辑」技巧，才可以让设计效益最大化。如果你想抓住下一个机会，那无疑是 3D 了。要知道，AI 在图像领域带来的效益其实是让「学习成本」以及「工作成本（例如时间精力）」下降，更多玩家愿意参与（不被学习难度吓坏），更多用户的需求被满足和催生。尤其是那些场景设计，费时又费力...

对于 3D 而言，学习成本更是巨大，即便你是一个老手，要构建精细可用的场景、角色等等都需要花费更多的时间。如果你关注 MR、关注 AVP（Apple Vision Pro），关注游戏制作、3D 软件开发。就会知道这些玩意儿有时候比平面更酷，它们可以被交互，带来更好更沉浸的体验。换而言之，我们不是不喜欢 3D，而是门槛太高，成本太大，入场太难了。

当 3D 的成本能够降低 2 个数量级会怎样？其中一层，就像 Midjourney 和 SD 引爆平面内容；同时很多新的内容平台应运而生（你可以看到很多 AI 内容平台）。就像手机摄像头的普及和进步之于 Tiktok 和抖音的意义。

让我们看看来自 @Yokohara 的最新探索，他使用 Tripo 在 10 秒钟用「单张图」生成模型，然后用法线贴图和视频混合的方式得到了下面这种惊人的效果 👇 你可以借此机会窥见未来 1 年 3D 带来的巨大威力。

如果你是内容创作者，我强烈建议从现在开始学习 3D 的基本编辑技能，提前做好迎接 3D 的准备；就像 Midjourney 悄然生长的时候，我开始现学各种新的设计技能。后来水到渠成，做 B 站教学视频的时候才没有感到什么制作上的困难。

总结下来未来可做的事情也就非常多了，Agent + 可交互的 3D 人物出现在你面前，那是怎样的体验？你可能知道 DAN 模式，但人物就在你身边会怎样？你可能想要学习科研，但导师就在你面前会怎样......从这个角度而言，3D 不是艺术设计，却能让你的开发为「空间计算」这个概念做好准备。

（现在是幻想时间 🤡...）

接下来是我最做的一个结合了 Midjourney + Magnific + Tripo + Blender 的简单工作流分享，适用于美术宣发。学习成本非常低，效果却很不错。要知道，新技术总是会在圈子内部被实验和火起来 🔥，外界的接收会延迟数月甚至数年的时间。先看效果图 👇

第一步：先用 Midjourney 生成一个机甲。

然后再用 Midjourney 生成不同风格的纹理贴图来作为 Magnific 的 Style Transfer Reference Images（风格参考图片）。我这里生成了 4 张，分别是：火、水、火星、迷彩风格。随后把 Structure Strength 拉到 100% 保证线条的一致性。

然后你就能得到这 4 种结果。

第二步：接下来就非常简单了，把图片作为输入直接给 Tripo 生成 3D 模型即可，免费就能试用。我这里喜欢用 Arc 浏览器多开窗口提高效率，哈哈。Tripo 类似于 Midjourney，会先生成初步结果（Draft），然后你可以对于喜欢的模型进行 Retry 或者 Refine（类似于 Upscale）。

但目前我比较喜欢第一阶段生成的 draft，因为 Tripo 的 Refine 新算法还没有更新，而 draft 的新算法上线，且效果很棒！（另外，draft 模型记得点击 ❤️ 爱心符号放到 Favorite 文件夹防止丢失。）

不过反过来说，这也挺好，要知道 draft 模型只需要几秒钟。在 3D 美宣的实际工作中其实已经足够降低 2/3 的工作时间了（来自一位 Blender 老法师的感受）。这位老法师和我们的做法有些不同，由于项目有更高的要求，他会把模型的每一部分拆解得非常精细，再用 AI 生成并重新组装（以及各种后续专业操作...）

接着，我下载模型并导入到 Blender 里面，增加了一层类似于清漆做法的材质，加个 HDRI 场景、打个光就 OK 了。如果你想得到一张更高质量的宣传图片，你还可以用 Magnific 的 Upscale 对图片进行放大。

最后加上 LOGO 就做好了。3D 的好处是你还可以绑定骨骼做动画，随意切换角度。如果你要利用 Video2Video，3D 作为视频的输入不会出现线条的错乱，大幅度动作的稳定性得到了保证。在这里，稳定性包括边缘线条、以及场景中的布光等等。视频教学点击这里查看。

本期分享就到这里啦！Catjourney 下周再回～