Configure DataConnect data stream

DataConnect data streams

DataConnect offer two data streams:

  • Native
  • Normalized

Native

Whenever the data is available on platform and DataConnect receives it - it will be immediately delivered to your RESTHook URL unprocessed, in the actual format which the platform is using.

Normalized

Whenever the data is available on platform and DataConnect receives it - it will go through the normalization process. Currently, it includes transformation from the platform’s native format to the schema.org format. It will then send it to your RESTHook URL.

You don’t need to care in which format data is coming. It will always be in the same format and with the same field names. You can find more details on schema.dataconnect.io page.

Configuration

No matter which eCommerce platform you are choosing or how you handle DataConnect data streams (see details below) configuration for data stream delivery is always the same and it includes the following settings:

  • Final Redirect URL
  • Fail URL
  • RESTHook URL

Final Redirect URL

After the last step of when a shop owner connects their shop with your app/plugin, DataConnect redirects to the "Final Redirect URL". That URL is pointing to your website where the new shop connection can be assigned to your user. To that URL we add a JSON Web Token ("jwt" parameter) which contains the connection ID. At that point you can decide how you assign it with the customer. Commonly, you will have the following use case.

Merchant found your application on marketplace, got interested and installed it. After technical integration happened he/she is redirected to your website (using "Final Redirect URL"). Your website is proposing to register or to log in (because it also possible that it is existing customer). After login or registration you assign delivered connection ID to that user.

Fail URL

This URL is only used when something is wrong during app/plugin connection process. As soon as we’ve got an error we will redirect user to that URL with a message explaining the reason ("error" parameter) of what happened.

RESTHook URL

That endpoint is used by DataConnect platform to deliver the data stream to your service. It is similar to Webhooks but it uses RESTful convention to deliver the data. You can check RESTHook website to get familiar with that way of data delivery.

Handling DataConnect data stream

DataConnect is implemented in a way that instead of you requesting the data from the platform - the platform delivers data to you.

You have two possibilities to handle data form DataConnect:

  • Using the DataConnect QuickStart App
  • Implementing data stream handling by yourself

DataConnect QuickStart App

The QuickStart App is a simple implementation of DataConnect data stream handling. It allows you to start getting data fast, without coding.

Self implementation

This is a more advanced and flexible solution because you are getting the data stream directly into your application. You can react on incoming data in near real-time manner, filter the data you want to process and store it .

Hints for easy development

As developers we know how hard it can be to integrate with 3rd party systems so we want to give you some hints which will make self implementation easier and fun by providing some info on the tools we also use.

Mockbin

If you want to quickly investigate which payloads DataConnect is sending without any effort - we suggest you use Mockbin. It’s an easy service which accepts any requests and logs everything including request methods, headers, payloads and URLs used for the requests. After you get your Mockbin URL you can set it as RESTHook URL, Fail URL, Final Redirect URL and proceed directly to the actual integration test.

Ngrok

During RESTHook implementation it’s really helpful when you can receive requests directly into running application on developer machine. It’s easy to test and debug implementation. To make it possible we recommend to use Ngrok. It’s a simple proxy which expose your application to the internet under some random URL like: http://12345.ngrok.com. You can set that URL in DataConnect developer portal as RESTHook URL, Fail URL, Final Redirect URL with necessary relative paths which is depends on your implementation and start working on integration without deploying your app to the server.

During RESTHook implementation it’s really helpful when you can receive requests directly into running application on developer machine. It’s easy to test and debug implementation. To make it possible we recommend using Ngrok. It’s a simple proxy which exposes your application to the internet under some random URL like: http://12345.ngrok.com. You can set that URL in DataConnect developer portal as RESTHook URL, Fail URL, Final Redirect URL with necessary relative paths. You can then start working on integration without deploying your app to the server.

Cons and pros

Any solution has advantages and disadvantages. Same goes to data handling.

QuickStart App

Pros:

  • No coding is required to handle DataConnect data streams
  • Supports both data streams (Native and Normalized)
  • Provides RESTful API for the data

Cons:

  • Not a near real-time integration; you need to request data on your own
  • You must use the tech stack which is used in the implementation
  • All incoming data is stored in the database

Self implementation

Pros:

  • Near real-time data stream which allows you to react immediately at the moment of the event
  • You can choose tech stack you want and to have implementation embedded in existing application or as a separate service
  • You are in control of what you are receiving and where you want to store the data and in which format

Cons:

  • You need to understand how data stream is delivered by DataConnect
  • You need to implement data stream handling on your own

Test

You are now ready to do a complete test for receiving the data from a connected shop. Please check connected platform guide for more details how to do an end to end test.