What is ElevenLabs? All the voice generation features to try

After guesting on multiple podcasts, I really admire the workflows and systems of the hosts. Production levels are high, and the final product always feels seamless. And tools like ElevenLabs are making that a whole lot easier. 

ElevenLabs keeps popping up on my feeds both from content creators in my network and from vibe coders embedding it into their work. It's arguably the best AI voice generation platform available, especially if you consider all the features it offers to take things up a notch.

So I spent time testing everything from creating dragon voices to building interview agents. Here's how it all turned out.

What is ElevenLabs?

ElevenLabs is an AI voice generator that focuses on emotional depth and context awareness. Unlike more basic text-to-speech tools, its voice technology interprets meaning, adjusts pacing, and delivers with appropriate emotion. (Usually.)

But it's more than just text-to-speech here. To show you the full gamut of features, I'll walk you through each of the experiments I ran, so you can see just how much you can do all on the one app. And yes, it borders on unnerving.

Voice Design

Voice Design lets you generate original voices using only text descriptions, without any source audio. You tell it what you want the voice to sound like, and it builds the voice for you.

I tested this with an ambitious prompt and some inspiration from Smaug the Tremendous: 

"An ancient, colossal dragon with a voice that reverberates through stone. This creature has lived for millennia with overwhelming superiority over all lesser beings. The voice is impossibly deep and resonant, spoken from vast cavern depths, with each word carrying centuries of weight." 

It didn't sound anything like a dragon.

But while I couldn't conjure Benedict Cumberbatch out of thin air using AI, I was able to create an audiobook narrator that worked comparatively better. By exploring existing voice data, I found a grandparently voice (the most popular English voice—Grandpa Spuds Oxley), which was pretty on-point. 

And there are plenty of other fun voices you can try in the Voice Changer tool.

No matter how you slice it, there's a learning curve. You need careful prompting to design a voice you like. But keep in mind that it has a 1,000-character limit (honestly, smart), so giving more context isn't necessarily the solution either. Be patient, and have fun.

Text to Speech

Text to Speech is the bread and butter here. ElevenLabs is great at converting text to natural-sounding speech with contextual emotion. 

I tested creating business content for LinkedIn, and I made sure to have some fun with it: 

"[pause] Here's something that completely changed how I think about productivity. [gulp] Last month, I discovered that 30% of my daily tasks could be automated with [pirate voice] simple workflows."

The voice interpreted the professional tone, added appropriate emphasis, and felt conversational rather than robotic. Have a listen.

Unlike in the previous test, simple action tags like [pause], [gulp], and [pirate voice] worked pretty well. I did notice that, depending on the ElevenLabs feature or voice I was using, sometimes action tags like "[pause]" were interpreted as words spoken aloud. My Maddy clone voice picks up the action tags just fine, but my dragon voice from Voice Design couldn't get it.

Voice Cloning

The free ElevenLabs plan includes Voice design, but paid plans get you access to additional features, including Voice Cloning. 

Instant Voice Clone lets you clone any voice (that you own) with as little as 10 seconds of audio, but it requires subscribing to the Starter plan ($5/month), at minimum. Of course I was curious about what my voice clone would sound like, so I gave it a try.

My first test was about 20 seconds of audio, and the end result was eerily close to what I sound like—if I only ever spoke with 100% confidence. It reminds me of Joey King's "Pretty" makeover in Uglies—something that feels uncannily perfect. I know you don't know what I actually sound like, but here's my voice clone.

I also tested the Professional Voice Clone feature. I had the perfect training data to use from my professionally produced LinkedIn Learning course recordings. With about a half hour of audio, my voice clone was sounding a lot more like me. Unnerving, honestly.

The cleaner the audio you provide (professional microphone, no background noise, etc.), the better the resulting cloned voice.

Note: I hope this goes without saying, but make sure that you actually have permission to clone the voice you want to use. If you create your voice clone with recordings, ElevenLabs will ask you to speak aloud to verify the voice in order to use it.

AI Dubbing and Translation

ElevenLabs can dub and translate content into about 30 languages while maintaining speaker voice characteristics—in theory. In practice, it was hit or miss.

I asked one of my colleagues at The Blogsmith to test the translation features for Hindi. He said the translation was on-point, but it made one error: it didn't use the gendered verb for the speaking woman that it should have. (I won't share the video because there are copyright issues since it's an actual ad.)

As you can imagine, when leaning on AI for translations (and in general), proceed with caution. Ideally, you should only use it to translate to a language that someone on your team is fluent in, so they can double-check the output.

Conversational AI agents

One of the most exciting features of ElevenLabs, in my opinion, is the customizable conversational AI agents.

You can start from scratch or with a template like Support agent or Sales agent. The idea is to create voice-enabled AI agents for phone calls and interactive experiences. 

I built an "interview agent" that would interview me about different topics conversationally to get the juices flowing for my content. All it took was a short prompt and a few tweaked settings.

I tested it by calling it. It's a very basic interface: you can start and stop the agent with a button, and the rest of the experience of interacting with it is voice-controlled.

Per my prompt, it introduced itself and asked what I'd like to talk about. It misunderstood my first request: I said something about "local LLMs," but it interpreted the speech as "local items." But I was able to fix it quickly, even interrupting it to make the correction.

The voice quality remained consistent throughout, but I'd definitely need to spend more time with the settings to make it as helpful as possible. As with any AI workflow, you'll need to craft prompts, test, and then tweak them until you get consistently good results. 

I'm using ElevenLabs' Conversational AI agents as a glorified voice-enabled chatbot, but you can create more robust workflows with advanced features like adding a knowledge base, giving the agent access to others apps via API, connecting using MCP, and integrating other functionality so the agent can actually take action.

Document-to-audio conversion

ElevenLabs can turn pretty much any content format into audio, which means you can go from document to podcast instantly. 

It's similar to NotebookLM's Audio Overview feature, turning your static documents into lively conversations between two podcast hosts. The main differences are that in ElevenLabs, you can edit the suggested script, and unlike with NotebookLM, you can choose the voices. It's a quirky new way we can use AI for more convenient content consumption.

I tested this feature with a copy of this article's outline, my voice clone, and good old Grandpa Spuds Oxley. 

I went with a short Conversation format, but you can also set it up as a Bulletin, which is a more focused podcast with a single host.

If you really dig this feature, check out ElevenReader—a sister product that turns books and articles into audio. 

As I'm writing this, the free tier includes two hours of listening per week, while the annual Plus plan offers 30 hours of listening per month, offline downloads, and a 12-month rollover for unused hours at $8 per month. For unlimited listening, check out the annual Ultra plan at $18.25 per month.

Speech to Speech

With the Speech to Speech function, you can record audio and then transform your voice—instead of just starting with text. It's a great way to capture nuances, laughs, and other subtle elements well. In my testing, it even retained specific accents and pronunciation from the original voice.

For example, here's a successful test transforming my voice to the dragon voice I'd designed earlier. I recorded myself taking a deep breath, starting to speak, then pausing for a laugh, and finishing the sentence. Here's the result.

Other audio tools in ElevenLabs

All the features I just walked you through make ElevenLabs worth testing, but there are a few additional tools that might seal the deal for you.

• Audio Native. Audio Native lets you embed audio players wherever you host your content. I previously used a WordPress plugin to convert text to speech on blog articles. The solution I was using is now defunct, so I'm glad to find what seems to be a great replacement. It works by connecting your domain to ElevenLabs, selecting a voice, and embedding the audio player on the pages where you want it to generate text to voice.

  

• Voiceover Studio is an intuitive user interface for professional voice production. If you're serious about voice design, you'll definitely want to give it a try.

• AI Voice Isolator helps you clean up recordings by removing background noise and leaving you only with the clear voice of the speaker.

• AI Sound Effect Generator essentially lets you create a custom soundboard for quick audio clips.

  

• AI Speech Classifier detects AI-generated (in other words, ElevenLabs-generated) voice content. I'm sure it's not 100% accurate (no AI detector is), but it can be a good start.

Integrate ElevenLabs into your vibe coding workflows

I first got to know ElevenLabs from a vibe coding context. You can bring the functionality and end results of ElevenLabs—both AI voice outputs and conversational AI agentic workflows—into other apps via API and a developer SDK. 

I used Bolt to vibe code a voice memo app that embeds ElevenLabs functionality via the SDK. The end goal was to be able to record my thoughts, generate a transcript, and take action on my notes—like sharing them in Slack or adding them as to-do items in Todoist.

To test it out, I decided to reference the interviewing agent I built with ElevenLabs.

When you press record, my embedded ElevenLabs agent listens and responds the way I taught it to. Then, after the interview ends, the app I built retrieves and displays the chat transcript for reference and further action.

ElevenLabs pricing

ElevenLabs offers a solid free tier, which includes Text to Speech, Speech to Text, Conversational AI, Studio, Automated Dubbing, and API access—all in limited amounts. You'll get 10,000 credits/month for free.

Credits are slippery because they're computed based on text length, the model you're using, and a voice rate multiplier (if relevant, based on charges set by creators). But generally, you can think of one minute of audio being about 1,000 credits.

If 10,000/month isn't enough, or you want access to other features, you'll move up to a paid plan. I'd argue they're pretty affordable for the value you get.

• The Starter plan includes 30,000 credits and costs $5/month. It also features commercial licensing and instant voice cloning.

• The Creator plan, which I'm currently using, includes 100,000 credits and costs $22/month. This gives me access to the Professional Voice Clone feature and overall higher quality audio.

Above that, most of the plans offer more credits for more money, and add features like even higher quality output, even via API, and team features.

Keep in mind that ElevenLabs no longer covers LLM costs, which will be passed on to the user and will impact the total credits required per task. You won't pay more than advertised, but you'll go through your credits quicker.

ElevenLabs is moving quickly

Even in the course of writing this article, ElevenLabs added new features and improved others. It really is a moving target, but it's only going to get better. And while my voice clone is fun for entertaining my husband, it's also changing the way I work. How will you use ElevenLabs for your marketing and customer service?

Related reading:

• The best AI video generators

• Make AI your new assistant

• The best AI marketing tools