Connectivity is one of the core challenges of automation. Think about the sheer number of apps your business uses. A recent Okta survey found that large customers deployed an average of 175 apps. And that’s not counting your legacy on-prem systems and custom builds. Getting all the apps you need hooked up to your automation platform is an ambitious and moving target. Not only is the total number of apps growing year over year, you’re also replacing apps over time as your needs change.
Generally, businesses tackle the problem of connectivity with a mix of three approaches:
- Built-in integrations or app marketplaces: these can be convenient but are often limited to one-directional data syncing or the most common use cases. In addition, built-in integrations offer point-to-point connectivity from one app to another. Complex automations involving multiple apps can be difficult to build without a centralized automation hub.
- Rely on your integration vendors: this approach works well for widely-used, high-value apps. For example, Workato is a leader in connectivity with more than a thousand connectors. However, the total SaaS market includes upwards of 30k apps, so you’re almost certain to need more connectivity than any iPaaS vendor can provide out-of-the-box.
- Build your own: building a custom connector gives you complete control, but with so many apps to connect, custom developments can easily become a bottleneck, and not every team is equipped for the task.
Building your own
In addition to an extensive library of Workato-built and community-built connectors, Workato provides powerful tools to make custom connectivity more accessible. For example, the universal HTTP connector can create a one-off custom connection to any platform with an HTTP API in minutes. However, there are times when you need something more permanent. For example:
- You want to build automations involving internally-built software, like a bespoke inventory management system.
- No pre-built connector exists for a third-party app that is central to your business processes.
- A prebuilt connector exists, but it doesn’t cover your use case. For example, you need to work with custom objects unique to your organization.
In these situations, the best way forward is to build your own connector.
Connector SDK
The Connector SDK allows you to connect any application, your way. With it, you can create a layer of abstraction over any API, to keep the end user’s focus on the business context. The end goal is an experience for the recipe builder identical to a Workato-built connector. The business user feels like they’re interacting with the application or service, not with an API. They don’t need to know about GET vs POST requests, access tokens, JSON or XML. The details of interfacing with the API can be solved by the development team, leaving the business user free to focus on the process being automated.
A detailed, developer-centric look at the Connector SDK is available in the Workato docs, but at a high level, you can use the SDK to:
- Authenticate to an app or service by any common method, including Basic, OAuth and API Key
- Subscribe to webhooks to create real-time triggers for event-driven automation
- Create polled triggers at customizable intervals, including batch triggers to process records in bulk.
- Create actions that a recipe builder can deploy to read or write data.
The SDK also includes tools to help the developer encapsulate complex actions to present the recipe builder with a simple experience for example:
- Built-in auto retry to easily create fault-tolerant automations
- Tools for working with bulk records, like UPSERT actions
- Data validation and error-handling tools
- Large file streaming
- Multi-step actions for calling asynchronous APIs.
Introducing Connector SDK Emulator & CLI
Until recently, all development with the Connector SDK has been fully cloud-native. Code is written, tested using the built-in debugger console, versioned, and released, all from within the Workato platform. This paradigm allows developers to rapidly develop, test, and release simple connectors. It also removes the need to manage dependencies, or a local environment.
However, as our customers and partners have grown, you’re using the Connector SDK to create mature, fully-featured connectors to tackle increasingly complex use cases. Since we started offering the SDK, your custom connectors have become:
- Larger – sometimes spanning 5000+ lines of code
- More frequently updated – as developers add new actions and refine existing functionality.
- More collaborative – with multiple developers working on a single connector
To support this maturing development community, we needed to provide more flexibility with regard to how you build custom connectors. To this end, we’ve released the Workato SDK Connector Emulator: a lightweight tool that allows you to build, debug and test custom connectors locally, work collaboratively with other developers, and manage change with your preferred Version Control System, just as you would with any other critical software project.
Unit Tests
Unit tests form the backbone of any sustainable development project. As your connector grows from MVP to a mature connector with thousands of lines of code, teams need to be confident that new changes will not cause regressions in the connector’s existing functionality.
Using the Emulator, you can now automatically generate test stubs for your connector, and implement full suites of regression tests, targeting individual components of the connector. As you continue to iterate, you can continuously run these unit tests to ensure that changes are safe. Tests use the common rspec framework.
You can also take advantage of VCR. This third-party library allows you to cache responses from an API and use them for testing. This means that your unit tests don’t require an active connection to run.
Easy Collaboration and Seamless Deployment
Managed your code completely within Github, or your preferred version control system. A dedicated CLI allows developers to create connector projects, easily commit them to a repository, and allow their teammates to work simultaneously in separate branches.
Each teammate can work on the connector locally, using the Emulator. Your team can review changes in Github and automatically run unit tests with your CI/CD tool to prevent regression.
When any code is approved and merged, the CLI can be used in Github Actions to seamlessly deploy the new connector to their Dev environment.
Enterprise-grade security
The CLI includes industry standard encryption features that allow you to encrypt and decrypt any sensitive information required for connector builds. You’ll be able to safely store encrypted credentials to APIs as well as any dependencies you’d need for your unit tests.