Webhooks
Replication your data from a variety of sources using our Generic Webhooks handler
Many platforms support sharing of data using Webhooks. Often, Webhooks provide real-time data as changes happen in the source system. Historical data is typically not available using Webhooks, but some sources may offer this capability. We suggest getting a manual data dump from the source to load historical data for sources configured via webhooks.
Sources may not always publish or honor the published documentation when it comes to data types. Therefore it is important to accurately detect or infer the data type based on the incoming data stream and create tables in the warehouse with the columns that have the most accurate data type.
A Webhook is essentially a way for an application to provide data or information to another application. It is a one-way data sharing that is triggered by events in the system. Typically, webhook end points receive data in JSON, CSV, or XML; JSON being the most common method. When a source is sending data to a Webhooks, it is possible that the data structure in which the data is shared changes over time. Columns may be added, columns may be removed, nested columns may be added, or removed, and many other such scenarios. It is thus important for Daton to gracefully handle these changes to the schema without the user having to worry about making changes to their pipeline.
Nested data when added to a webhook data stream must be handled efficiently. Depending on the warehouse, the nested data may have to be flattened, or loaded as a JSON in the warehouse.
Daton supports Append-Only mode of loading for Webhook data. Although primary keys may be available in the source, the data in the warehouse will only be appended to the tables configured using the webhooks. User must de-duplicate the data prior to using the data for reporting or analytics needs.
This integration name will be used to identify the Webhook in the Daton integrations screens.
Enter a table name. Daton will use this input to create a table in the warehouse. Please make sure that the table name is unique and not already in use in the data warehouse.
Daton auto-generates a URL that can be used to configure the Webhooks. Use the Copy to clipboard functionality to copy the URL and use it in your code or within the source application.
If you have any questions about setting up Webhooks, please reach out to us on Chat or email us at [email protected]
Last modified 1yr ago