Need to work with large files in your Custom Connector or API Platform? We have you covered with a set of capabilities that makes it easier to work with files. We are also excited to continue to expand our regional availability by launching the Japan data center.
To see all the newest updates in Workato, here is our roundup of releases from October.
File Streaming support for Custom Connector SDK
When working with files, it’s simple to imagine that files can be easy to move around when they are small PDFs or CSVs. However, things get trickier when you consider real-world enterprise use cases like moving gigabytes of CSV data for a year’s worth of invoices or a huge video file.
Most of these require complex API orchestration to be able to efficiently move this data in smaller chunks from source to destination, which can be challenging. In Workato, we’ve solved this problem through a simple abstraction that we call file streaming.
What is file streaming?
File streaming is the concept of reading and writing a file in smaller parts (chunks) in sequence. This allows us to transfer large files between apps or file systems without worrying about hitting the size limit of the sending and receiving systems.
It is initiated when users map a single datapill representing file contents – regardless of size – from their desired source app to their desired destination app. With this simple map, we handle the complexity of orchestrating this transfer for users.
Today, file streaming is available for most of our file connectors like Azure, GCS, and S3 as well as top Saas applications like Marketo, Salesforce, etc.
File streaming in smaller chunks
File streaming with Custom Connector SDK
As we continue to invest in the Connector SDK, this month we have added File Streaming. Now, the same file streaming power is available as part of the Custom Connector SDK.
This enables anyone to build a connector using APIs that allows for the same, awesome user experience as with our native connectors. By removing the previous file limits, connectors can now stream large files upwards of 10GB, enabling you to automate more.
To learn more, check out the How-to guide for File Streaming.
Large File support for API Platform
The API Platform is a great way to turn API Recipes into your own endpoints for others to use. Using files as part of your recipes is another way to get more out of the API Platform for data load or data synchronization-related use cases. However, sometimes files can be quite large and can create issues when used.
For example, when submitting a job application on a web portal, candidates need to fill out a form and attach their resumes. This form is then submitted to the backend API as form data with multiple parts (text values along with a file).
Another example is working with large images or video files for automations that could be used to help manage marketing assets for external communications. Working with large files should be just as easy as any other file and Large File support addresses this.
Addressing large files
To help solve this, a new content type of “multipart/form-data” has been added for requests and a new type of “Attachment” has been added to the response for easy reference.
Multipart form type
Related: Custom API Security: How to use custom API calls securely
New Japan Data Center
Workato expands further into Asia Pacific by launching a Japan data center – here are its benefits.
- Helps support the accelerating growth and adoption of the Workato platform amongst our growing base of customers in Japan.
- Ensures that customers have more control over where their data is stored and processed
- Customers are better able to comply with the data protection regulations, as well as satisfy the internal data protection and security needs of their organization.
As part of our effort to better serve customer needs for compliance and preference for where the data is stored, the new data center will process and store the data within the Japan data center. This provides additional security and protection to our customers who prefer their data to stay within Japan. To learn more, check out Workato’s Supported Cloud Regions
Japan data center