A blog about productivity, workflow best practices, company building and how to get things done with less work.

 

Archives for November 2012

I won't rag on Unleashed too much as their API is in beta, but there are a few problems that exhausted a lot of my time that I thought I'd cover in a quick post. If you'd like the quick version of my advice:

Don't use their JSON API yet (11/25/2012), suck it up and do XML.

First of all, you'll need to generate your own UUID's. This isn't particularly hard (we use Python's import uuid library), but did throw me for a loop at first. Also, the guid's you generate need to be put in both the URL and the XML you POST.

Next, the JSON formatting of dates is very bizarre (at least at the time of writing). They opt for \/Date(1331550000000)\/ which I assume is milliseconds from epoch? Further, their JSON API doesn't serialize errors very well (read: at all). You'll see a lot of 403: Bad Request with no further details provided. However, their XML API does just fine! So, don't use JSON.

Next, some keys they say are required, aren't (and some they say aren't required, are). The best way around that is to do a few sample POSTs with more or less empty XML. Then you can start adding elements as they return missing key errors. (Be sure to use XML, not JSON!)

Finally, they also let you define related sub elements by various identifiers. For example, you don't need to include the guid of a product: you can alternatively include the code or name instead.


About the "API Quirks" Blog Series

Everyone at Zapier has to deal with API Quirks. We deal with so many APIs each day, we have become almost immune to the often bizarre, undocumented, frustrating behavior exhibited by web APIs. You can see all the posts in this series here. No API or documentation is static. Take any quirks published before "today" with a grain of salt.

Don't want to worry about this? Connect to Unleashed with Zapier

About the Author

Bryan Helmig is a co-founder and developer at Zapier, self-taught hacker, jazz/blues musician and fine beer and whiskey lover.

Zapier fully supports webhooks, and though they can be a little overwhelming at first they can be extremely powerful. The easiest way to think of webhooks is as notifications. At their simplest, they carry a payload of data which is usually a single record that has been created or modified, although some services will send along multiple records at once in the name of efficiency.

There are three common serialization formats that you will commonly see: form-encoded, JSON and XML. Below are examples of each, respectively:

first_name=Bryan&last_name=Helmig&age=27

{
    "first_name": "Bryan",
    "last_name": "Helmig",
    "age": 27
}

<first_name>Bryan</first_name>
<last_name>Helmig</last_name>
<age>27</age>

It is important to be familiar with each, as most services have different standards though some have options, so you can experiment a bit. Don't worry too much though, Zapier handles all of them just fine!

Once you have a payload, you need somewhere to send it. Usually its to a special URL where a server is waiting to interpret the payload and do something with it. Usually that is a server you control, but in the case of Zapier, its a server we control on your behalf. When you create a new Zap with a webhook enabled trigger, we will generate a brand new URL for your use, sort of like this one:

https://zapier.com/hooks/catch/n/Lx2RH/

Once you have the URL, you'll be able to provide it to the triggering service. After you do that, we should start receiving payloads whenever the service sends them.

Also, Zapier can do the inverse, you can give us a URL and we can push a payload to it. But more on actions later, first, let's keep talking about triggers.


Debugging Triggers


We do our best to just make webhooks "just work" for you, but we can't predict all the wacky ways services might choose to implement webhooks. Sometimes, you just have to break out a few debugging tools and see what is going on under the hood.

The simplest place to start is Zapier itself. After you've started your Zap and have configured the webhook URL, we start listening. Even if you don't unpause your Zap, we still track the payloads we get to make it easier for you to debug. You can see them by clicking "Load Samples" on step 4, and we'll also pull keys into step 2 to make it easier to map. Here's how to make sure you see them new payloads:

  1. Create your Zap and configure your webhook URL but don't enable your Zap quite yet!
  2. Go into the triggering service and do something to trigger a webhook (usually that means create a record of some sort).
  3. Return to Zapier and click "Load Samples" or "Reload Samples" in step 4.
  4. Do you see the new values coming in? Congrats! You can finish editing your Zap and enable it.

If you didn't see the samples, then we need to do a little more debugging. Let's try the next step.

Generating Your Own Requests

Sometimes, you'd like to trigger Zapier webhooks on your own without involving a third party service. There are a few tools that help you do that (like curl) but we generally recommend a service like hurl. If you'd like to use curl, check out this handy field guide.

When using curl or hurl.it, we work best with POST requests with JSON payloads. Here's a quick curl example to get you going:

curl -v -H "Accept: application/json" \
        -H "Content-Type: application/json" \
        -X POST \
        -d '{"first_name":"Bryan","last_name":"Helmig","age":27}' \
        https://zapier.com/hooks/catch/n/Lx2RH/

Inspect the Requests

Next, we need to see if the webhooks are even getting sent, or what sort of weird format they might be in. For that, we like to recommend a service like Requestb.in. Requestb.in doesn't try to interpret the payloads, it just prints them to your screen as it receives them, making it easy to see what is going on. Just click the "Create a Requestb.in" button on their homepage and copy the link in the top bar, it will look kind of like this (make sure it does not have ?inspect at the end):

http://requestb.in/17e2uvw1

Now, log back into your triggering service's settings and replace the old Zapier webhook URL with the new Requestb.in URL. Then, take the steps necessary to trigger the webhook once again (usually creating a record of some type). After you do that, simply refresh your Requestb.in page and take a look.

Now the question is: did you see any new records show up?

If yes, that means Zapier is getting the payloads but couldn't intepret them properly. You should contact us and send us your Requestb.in findings or, if you are a programmer, you can always try parsing the payload yourself via our Developer Platform.

If no, that means the service isn't sending the payload! Usually this is just a misunderstanding as they may not have webhooks enabled for the records you want, but sometimes you just need to check another checkbox somewhere, so you might trying contacting that service before contacting us.


Debugging Actions


Like we mentioned earlier, Zapier also can send webhooks, not just receive them. That makes it easy to pipe data around the web, especially if you don't want to muck around with setting up connecters to various APIs: you can just dump the data in a nice format via Zapier.

Sending webhooks via Zapier is much more complicated than receiving them via Zapier as you are in charge of interpreting the payloads yourself. If you are a programmer, this usually isn't that hard. If you aren't a programmer, hopefully we can help you as much as we can, so let's get started.

Inspecting Requests, Again

Just like debugging triggers, we'll be using Requestb.in to inspect the payloads. This will help you understand what Zapier is sending, making it easier to adjust it to your liking. First, choose any service as a trigger, though it makes sense to choose one you are familiar with, it certainly isn't a requirement. Second, choose Web Hook as your action, POST is probably the most popular, so start there.

In a reversal, instead of Zapier giving you a URL, you'll need to give Zapier a URL. So, give us a Requestb.in URL by visiting their homepage and clicking "Create a Requestb.in" and copying the URL at the top of the screen (make sure it does not have ?inspect at the end) and paste it into Zapier's Webhook URL field when you are editing your new Zap. Again, they look kind of like this:

http://requestb.in/17e2uvw1

Next, you'll need to start configuring your payload. We offer 4 different options at the time of this article: Send as JSON?, Data, Basic Auth, and Headers. Let's cover a few examples (leaving the rest empty for now):

  • Send as JSON?: yes
  • Data hello|world

Will result in this webhook (default headers are up top, payload at the bottom):

Content-Type: application/json

{"hello": "world"}

Now, let's say you switch Send as JSON? to no. Now, you'll get this webhook:

Content-Type: application/x-www-form-urlencoded

hello=world

That certainly isn't the most useful, as the data is static: it never changes. Let's imagine a better example: perhaps we have a Dropbox trigger that pings our web server telling it that there are new files to be copied. We'd want to know about the new files, so maybe we'd do something like this:

  • Send as JSON?: yes
  • Data: path_to_file|{{path}}

Now, the {{path}} is done via dragging fields from the left to the right in our Zap editor interface. All you have to know right now is that we'll replace it with the real path on the fly, like this:

Content-Type: application/json

{"path_to_file": "/Important Documents/Business Plan.txt"}

Of course, you aren't limited to provided only a single item in the data, plus, you can add headers in the same way like so:

  • Send as JSON?: yes
  • Data:

    path_to_file|{{path}}
    file_size|{{size}}
    hello|world

  • Headers: X-From-Service|Zapier

Which will result in a webhook like this (again, headers are up top):

X-From-Service: Zapier
Content-Type: application/json

{"path_to_file": "/Important Documents/Business Plan.txt", "hello": "world", "file_size": "822 bytes"}

Another option we didn't cover yet is the ability to leave data empty, which tells Zapier to send along all the raw data from the trigger side (which you can always see in step 4). This is a special case that may save you from maintaining a giant list for data mappings.

*Pro tip! * If you want to get even fancier, you can define children objects with our handy double underscore syntax, for example, this:

root|value
child__extra|value
child__other|value

Will turn into JSON that looks like this:

{"root": "value", "child": {"extra": "value", "other": "value"}}

Catching the Webhooks

Actually catching and interpreting the webhooks we send is another matter. If you are trying to use webhooks to send payloads to a service we do not support, you may run into insurmountable issues as our data field doesn't support complex structures (for example, lists). In those situations, we highly recommend investing a little time in our developer platform.

However, if you are the one writing the code to catch the webhooks you should be able to interpret with only the examples above as a guide. With PHP, the easiest is probably form encoded (Send as JSON? as no):

<?php
echo $_POST['path_to_file'];
echo $_POST['file_size'];
echo $_POST['hello'];
?>

But, if you set Send as JSON? to yes, you'll need to decode first:

<?php 
$data = json_decode(file_get_contents('php://input'));
echo $data['path_to_file'];
echo $data['file_size'];
echo $data['hello'];
?>

About the Author

Bryan Helmig is a co-founder and developer at Zapier, self-taught hacker, jazz/blues musician and fine beer and whiskey lover.

It's been a crazy few months since we publicly launched in June. After lots of great feedback from users, we are happy to announce all new plans and pricing. Specifically: a better free plan.

We sat down, talked to many of you, and asked how we can deliver you the best bang for your buck. Our new plans are the direct result of those conversations. Additionally, our new plans will enable us to grow Zapier with steady revenue so that we can continue to deliver you a killer product and customer support experience into the future. Here's the details:

What Are These New Plans?

To start, the free plan is getting way better and of course, it's still free.

The new free plan:

  • 15 minute syncing. That's right, 15 minutes. No more crappy 5 hour syncing periods.
  • Access to all standard services.
  • As many connected service accounts as you want.
  • 5 Zaps.
  • Up to 100 tasks a month (with ways to earn more).

What's different? To start, the syncing interval is 20 times faster. You'll have 5 Zaps and at least 100 free tasks/month, but now have the ability to earn more. You can have as many inactive Zaps as you'd like and connect as many service accounts as you want.

What About the Paid Plans?

We have three paid plans available (and enterprise plans, contact us). All the paid plans come with:

  • 15 minute syncing or faster
  • 20 zaps or more
  • Access to all services (standard + premium)
  • 3000 tasks/mo or more
  • Premium support
  • Priority for feature requests for new services, triggers, and actions
  • Unlimited high fives and free t-shirts if you come visit us in Mountain View, CA, Columbia, MO, or Chicago, IL.

What's Next at Zapier?

We want to help you work more efficiently. Some tasks are best for a computer and you shouldn't have to be a programmer to make that happen. Zapier will do the work for you -- you've got more important things to worry about.

We will continue adding new services and working with third party developers and vendors to make sure Zapier supports any services you use.

We're also working on some new features and a better user experience. This stems directly from the feedback we get from users.

Is there something you want to see Zapier support? Tell us! Send us an email or comment below.

Answers to Questions you might have

I'm on Free, what happens to me?

You can continue to use the free plan you signed up for or upgrade to the new free plan. If you are using premium services, you can keep using premium services on the old free plan, but as soon as you switch to the new free plan, the premium services won't be available unless you join a paid plan.

If you're worried you might make a mistake. Don't be. We're just a quick email away and we can fix anything that you might do to your account.

I'm on Basic, Business, or Business Plus, what happens to me?

You can continue using the plan you signed up for or switch to any of the new plans. If you like a new plan better, you can switch to that. Once you switch there isn't a method to switch back to the old plans so if you switch, make sure that is what you want.

If you're worried you might make a mistake. Don't be. We're just a quick email away and we can fix anything that you might do to your account.

If you feel the Free plan is more suitable for you now, you can of course cancel and grab the new free plan instead.

P.S. You Can Earn More

If you are really enjoying Zapier on the new free plan, you can of course Go Premium to automate more tasks or check out some easy ways to earn free tasks.

After chatting with hundreds of you, we are certain this will make your lives better with Zapier. If you have feedback for us just drop a note in the comments or emails us any time at contact@zapier.com.

About the Author

Wade Foster is a Co-founder and CEO at Zapier. He likes to write about process, productivity, startups and how to do awesome work.

Get help