Stable Diffusion and Midjourney are two of the best AI art generators currently available. They both allow you to experience the bizarre magic of entering a text prompt and watching it generate matching images in seconds.
But among a sea of AI art generators (which, unlike these two, are mostly based on other people's AI models), Stable Diffusion and Midjourney are two of the most powerful and customizable options available.
If you're trying to decide which of the two top image generators to use, read on.
Midjourney vs. Stable Diffusion at a glance
Stable Diffusion and Midjourney are similar tools with a few key differences. Here's a short summary, but read on for the full rundown of how they differ.
Midjourney | Stable Diffusion | |
|---|---|---|
Quality of images | ⭐⭐⭐⭐ Some of the best AI-generated images, though may not fully understand every prompt | ⭐⭐⭐⭐⭐ Arguably the best AI-generated images right out of the box |
Ease of use | ⭐⭐⭐ You can only access it through Discord, which is a pain, but there's a web app coming | ⭐⭐⭐⭐ Stable Diffusion is available in so many ways—some of which are super easy to use |
Power and control | ⭐⭐⭐⭐⭐ Best-in-class prompting and editing options | ⭐⭐⭐⭐⭐ Powerful from the start, with the ability to train models on your own data |
How do Stable Diffusion and Midjourney work?
Stable Diffusion and Midjourney work in much the same way to create images—it's the user interface and various other features where they differ, not the underlying technology.
Both AI models were trained on millions or billions of text-image pairs, which is how they can understand what concepts like people, Canada, and impressionist paintings mean—and then put it all together to interpret a prompt like "an impressionist oil painting of a Canadian man riding a moose through a forest of maple trees."
Once Stable Diffusion or Midjourney has worked out what it thinks you're asking it to create, it generates an image using a technique called diffusion. The image generator starts with a randomly created field of noise and, in a number of discrete steps, edits the image to get it closer to its interpretation of your prompt.
Every time you generate a new image, the image generator starts with a different field of noise, which is why you can get different results, even if you use the same prompt. In a previous article comparing DALL·E 3 and Stable Diffusion, I described the process as "kind of like looking up at a cloudy sky, finding a cloud that looks kind of like a dog, and then being able to snap your fingers to keep making it more and more dog-like."
While that's not a perfect analogy, you can tell by the fact I'm reusing it here that I think it's a good way to think of it.
Even though the underlying tech of both models is pretty similar, both Stable Diffusion and Midjourney can produce completely different results. How a model interprets your prompt and weighs up all the different elements of it, the specific data it was trained on, and the underlying philosophies of the team developing it all have a huge effect on the final output.
Let's use one of my favorite prompts. Here's what Stable Diffusion (through DreamStudio) created for "an impressionist oil painting of a Canadian man riding a moose through a forest of maple trees":
And here's what Midjourney made:
While both sets of images look pretty like impressionist paintings, Stable Diffusion has done a much better job of including a moose. With Midjourney, some of the images have pseudo-antlers, but it really looks more like a horse.
Still, there's a lot more to explore, so don't call this one for Stable Diffusion just yet.
Stable Diffusion is available in more ways; Midjourney is still only available through Discord
What I'm calling Stable Diffusion is actually a number of open source models. At the time of publishing, the latest version is called SDXL 1.0. And there are lots of ways to access it:
For this article, I'm accessing it through Stability AI's DreamStudio app, which I'd argue is the best way to do it.
There's a free, even simpler way to use it through Clipdrop if you're happy to use the slightly less capable SDXL Turbo beta.
Or, if you want to go all in, you can download the source code, install it on your own computer, and even train an AI image generator on your own data. This is how many services like Lensa's AI avatars work, and how other tools like Canva integrate image generators into their apps.
The huge variety of ways you can use Stable Diffusion makes it suitable for lots of different uses. Need to throw together something quickly? Use Clipdrop. Want a bit more control? Try DreamStudio. Building your own startup that sells people drone images of their houses with gigantic monsters added in? Grab the source code, and train your own model.
Midjourney is not like this. Right now, it's available in just one way: through Discord. Yes, one of the best AI image generators available right now is only available through a chat app (granted, a great one). And while it's not hard to sign up for a Discord account and join Midjourney's server, it is still an odd way to access an AI art generator.
The good news is that the Midjourney team is developing a web app, which will give it a nice, logical interface. It's not available yet, but when it is, you won't have to prompt the Midjourney Discord Bot by typing /imagine <your prompt> in a direct message or public channel.
Even when that happens, though, Midjourney will be easier to use, but you still won't be able to integrate it with your own apps and services.
Both Stable Diffusion and Midjourney are really powerful
Both Stable Diffusion and Midjourney are really powerful—just in different ways.
In every version except Clipdrop, Stable Diffusion allows you to set an image style, add negative prompts for the things you don't want to see in your images, use an image as part of a prompt, and do things like set the prompt strength, the number of generation steps the model takes, and even the exact random seed it uses. And that's before you get into training it on your own data, which is really where Stable Diffusion is in a class of its own.
Midjourney, on the other hand, is just as powerful but in a more personal way. Out of the box, it gives you a huge amount of control over what exactly the image generator creates—though for most features you need to add a parameter to your prompt by typing -- followed by the one you want to use. It's a tad awkward, but you can do things like use multiple images as part of your prompt, blend aspects from all of them, set the aspect ratio, control how weird or varied your images are, and set what random seed you start from. (You can find a full list of parameters here.)
But really, it's the tools that allow you to upscale, vary, and edit your images and the ease with which you can use them that differentiate Midjourney from Stable Diffusion.
While Stable Diffusion allows you to modify your prompt, create variations on any of your generated images, or upscale them to 2048 px wide, Midjourney takes things further: you can upscale your images up to 4096 px wide, choose whether you want to create subtle or strong variations, and even select specific parts of the image to change, all while tweaking your prompt if you have Remix mode turned on.
Although Stable Diffusion (at least in DreamStudio) technically has an edit feature that is meant to allow you to extend your creations beyond their boundaries, I've never had much success with it. It's tricky to use, and the results have never been great. Midjourney, though, nails it. The Zoom Out and Pan options automatically extend your images in your chosen dimension, and when you combine them with the other tools, you can really have a lot of success.
Both can make awesome AI-generated images
When it comes to overall image quality, it's really hard to make any sweeping statements declaring a clear winner. Stable Diffusion—at least through Clipdrop and DreamStudio—is simpler to use, and can make great AI-generated images from relatively complex prompts. I find it's better able to parse longer, more nuanced instructions and get more details right.
Midjourney, though, gives you the tools to reshape your images. Even if it doesn't get things perfect the first time, you can make changes and get your images to the place you want them to be. I find it's the AI image generator I turn to most often because of it, and overall, I like its default image style more than Stable Diffusion's. But it's all subjective.
Both Stable Diffusion and Midjourney have their quirks
While Clipdrop and DreamStudio are relatively easy to use, getting the most from both Stable Diffusion and Midjourney requires more than just clicking a few buttons.
With Stable Diffusion, if you really want to use it to its full potential, you'll need to look at getting it up and running locally on your own machine or through a cloud provider like Leap AI, and training it on the kinds of images you want it to create.
And Midjourney? Well, at the moment, you can only use it through a chat app—and all your images are published to Midjourney's member gallery where everyone can see them, unless you subscribe to the $60/month Pro plan and activate Stealth Mode. Everything from creating images and selecting which model to use to changing your profile settings is handled through Discord commands, which is kind of ridiculous. Again, a real web app is coming at some point soon, which will smooth out some of these quirks, but presumably not all of them.
As long as Midjourney keeps all its editing features, when the web app launches, it will probably be the best AI image generator for most people. For now though, it's Discord time.
Stable Diffusion probably wins on price
Given that Stable Diffusion is usable in a few different ways, pricing is a little all over the place. You can use SDXL Turbo, a real-time version of SDXL, through Clipdrop for free, though you only get one output at a time, and they're watermarked. For $13/month, you can use SDXL to generate 1,500 images per day.
DreamStudio uses a credits system, though it isn't exactly one credit equals one prompt. Instead, the different settings you use all affect how many credits it costs to generate each image. If you generated four 1024x1024 px images with the latest model using 50 steps, it would cost 1.01 credits. If you used just 30 steps, it would only cost 0.8 credits. Bump it up to 100 steps, and it will be 1.56 credits. (You can always see the cost before you press Dream.)
When you sign up for DreamStudio, you get 25 free credits, which is enough for ~100 images (or ~ 25 text prompts) with the default settings. After that, it costs $10 for 1,000 credits. That's enough for more than 5,000 images or ~1,250 text prompts at the default settings.
And then, of course, you can download and run Stable Diffusion for free on your own computer—though if you want to use the images you generate for commercial purposes, you'll need a Stability AI Professional membership, which costs $20 per month.
With Midjourney, things are a little simpler. There's no free option, but the Basic Plan starts at $10/month and entitles you to 200 minutes of GPU time. How many images is that? Well, Midjourney says it's good for roughly 200 prompts, but creating multiple variations or upscaling your images will burn through them faster than creating lots of low-res images. To make things a tad more complicated, starting with the $30/month Standard plan, you get enough fast GPU hours for roughly 900 prompts per month, but you can also generate unlimited images in Relax mode—which only runs when there's free GPU power.
Once again, it's hard to make any grand conclusions, but in general, Stable Diffusion offers more image generation for less cash. Plus, you can at least try Stable Diffusion for free, whereas you have to pay at least $10 to try Midjourney.
Commercial use continues to be complicated
Both Stable Diffusion and Midjourney allow you to use any images you generate commercially with very few limits, but that doesn't mean that things are simple. We're in a new era, and a lot of lawsuits are still working their way through the courts to determine what exactly is and isn't legal with AI-generated art.
Back in February 2023, the U.S. Copyright Office ruled that images created by Midjourney and other generative AIs can't be copyrighted. This means that even though you're free to use images you generate commercially, you have limited recourse if someone takes them and uses them without your permission. So, it's important to remember if you're designing logos or otherwise building a brand using Stable Diffusion or Midjourney, you don't have a very strong legal shield.
Generally speaking, Stable Diffusion probably wins for commercial use. The flexibility to train your own models and its ability to better understand complex prompts just makes it easier to fit into a professional workflow.
Stable Diffusion vs. Midjourney: Which should you use?
This isn't an easy choice. And of course, it doesn't have to be a choice—you can always use both.
If you want the best images with minimal extra prompting or want to train your own model, Stable Diffusion is the one to go with. It's also the only option with a free trial. But if you're happy to edit and work with Midjourney, you're likely to get the best possible AI-generated images—it remains my go-to, despite Discord being really weird.
Alternatively, if you want the easiest option going, check out DALL·E 3. It's a bit less powerful than either Stable Diffusion or Midjourney, but it integrates with ChatGPT, so it's really easy to use.
Related reading:
