Skip to main content

How do I create data feeds?

Updated

The method used to create data feeds differs depending on the source data you’re connecting to. A data feed can be connected to data from many diverse sources, for example, data from services (like Google Analytics and HubSpot) or Excel spreadsheets, or data that’s retrieved via a REST/URL query. This article explains when and how to create data feeds in a general way and includes links to more detailed articles for each type of source data.

This article includes:

Notes:

When do I create data feeds?

You’ll create standalone data feeds or create data feeds as you build custom metrics:

  • This article focuses on creating standalone data feeds that you later use to build custom metrics.
  • You can also create data feeds as you build custom metrics. Creating a data feed is an integral part of the custom metric creation process. Learn more about creating custom metrics and their data feeds here.
  • Note that when you add instant metrics, you don’t need to create data feeds. Klipfolio automatically creates and manages those data feeds for you.

How do I create data feeds?

The method you use to create data feeds differs slightly depending on where the data is coming from, for example, if you’re connecting data feeds to data from uploaded spreadsheets versus connecting to data in your Google Analytics account.

The general workflow is:

  1. Choose the data you want to connect to. This might be uploading a spreadsheet, entering credentials for your Google account, or pinpointing the desired data to retrieve using a query.
  2. Refine the data in the data feed editor. No matter where your data is coming from, you end up in the data feed editor where you can modify the data or save it as-is. You can always return to a data feed later and make adjustments, from superficial changes to starting over by connecting to a different source of data.

To create a new data feed:

  1. Regardless of the service you’re connecting to, you begin in the left navigation sidebar, where you click the + button beside Data Feeds to add a new data feed. (See below.)
    add data feeds button
    This opens a new, empty data feed.
  2. Click Select data and, on the Where is your data? page, choose the service you want to use for the data feed.
    The next steps differ depending on the chosen service. Refer to the relevant section in this article, as listed below.

The following section includes general information (and links to more detailed information) on:

Creating data feeds for services that support instant and custom metrics

Our most popular services can be used to create instant and custom metrics. This section describes the general method for creating data feeds related to those services. You can also read specific instructions for each service by clicking the links below.

Services that support instant and custom metrics include:

The method for creating data feeds differs slightly depending on which of the above services you choose, but for the most part, the workflow looks like this:

  1. Connect to the service. The first time you create a data feed for a service, you select the service, choose your account and, sometimes, account settings (for example, you enter your Google Ads login credentials and your Google Ads customer ID). The next time we’ll remember the last-used account and settings to save you time.
  2. Choose the data to retrieve. Next, you enter the query builder, where you define the data you want to retrieve for the data feed. To do this, you’ll choose a data view and one or more columns, and, optionally, apply filters. Preview the queried data and make changes until you’re happy with the result. Click Save query.
  3. Finalize and save the data feed. Lastly, you enter the data feed editor, where you can save the data as-is, in which case you’re done! (Click Save data feed.) If you want to make some changes, you can go back to the query builder and make adjustments to the query, or stay in the editor and modify the data there. When you’re satisfied, click Save data feed.

Creating data feeds for core services

Our core services include:

The method for creating data feeds differs depending on the core service. General descriptions for each core service are described below, including links to more detailed articles.

File upload

When creating data feeds for uploaded files, you locate and upload the chosen file. Next, you enter the data feed editor where you can either save the data as-is or make modifications. When you’re happy with the data feed presentation, save it. It’s automatically added to your list of data feeds, from which you can modify it or use it to create custom metrics.

Note: We support the following file formats: Excel, XML, CSV, and JSON.

Learn more here.

FTP & SFTP

When creating data feeds for data that’s stored on FTP or SFTP servers, you configure the connection to the server and select the spreadsheet you want to connect to the data feed. Next, you enter the data feed editor where you can either save the data as-is or make modifications. When you’re happy with the data feed presentation, save it. It’s automatically added to your list of data feeds, from which you can modify it or use it to create custom metrics.

Note: We support the following file formats: XML, RSS, CSV, Excel, and JSON.

Learn more here.

REST/URL

When creating data feeds for data that’s retrieved using a REST/URL query, you enter an API query according to the web service’s API specification, select the output format (Excel, CSV, JSON, or XML) and the method (GET or POST), set query parameters, and configure the authentication (if applicable). Next, you enter the data feed editor where you can either save the data as-is or make modifications. When you’re happy with the data feed presentation, save it. It’s automatically added to your list of data feeds, from which you can modify it or use it to create custom metrics.

Learn more here.

SQL query

When creating data feeds for data that’s retrieved using an SQL query, you connect to and query an SQL database. Next, you enter the data feed editor where you can either save the data as-is or make modifications. When you’re happy with the data feed presentation, save it. It’s automatically added to your list of data feeds, from which you can modify it or use it to create custom metrics.

Note: We support the following JDBC drivers: MS SQL (Microsoft SQL Server), MySQL, Oracle Thin, Oracle OCI, Sybase SQL Anywhere, PostgreSQL, Firebird, and DB2. ODBC drivers are not supported.

Learn more here.

Creating data feeds for all other services

“All other services” refers to non-core services and legacy data services for which you can create custom metrics only (not instant metrics). You can find specific instructions for these services by searching in the Knowledge Base by the service's name.

Creating data feeds from JSON/XML source data

When your source data is in a JSON or XML format, the method of processing it in the data feed editor is slightly different than when dealing with source data that’s already in a tabular format. In the data feed editor, you can quickly populate columns with specific data elements from the tree-structured source data and then add context to your data using formulas and properties.

To create a data feed from JSON/XML source data:

  1. In the data feed editor, click the + button to add a column to your data feed table.
  2. Name the column. This is how it will be referenced when building custom metrics. Each column is automatically named according to the first value in the data selection.
  3. In the tree-structured source data, click the black arrows to show or hide the elements of each data node.
  4. Select the element of data you want to use to populate your column. All elements of data with the same name will populate your column. For example, if you select “type”, all elements of data with the name “type” will populate your column.
  5. To further specify the data assigned to each column, you can use the Selection Options or edit the XPath in your formula bar:
    • Selection Options: When an element of data is selected, you can click the Selection Options link on the right side of the data. Here, you can choose from a list of varied selections to specify the data you want to see in your column. When you click on an option, you can see a preview of what’s being selected.
    • XPath: You can use the formula bar to write a specific XPath to ensure that the correct data is being selected for each column. To do this, you select an element of data and edit the XPath by double-clicking on the path in the formula bar or clicking in the formula bar to add your cursor after the path and pressing backspace on your keyboard. Edit your path between the @ and ; symbols in the formula bar.
  6. Add context to the columns by setting the data types, specifying data formats, and writing formulas.
  7. Click Save data feed.
    Learn more about JSON and XML data.

I’ve created some data feeds - What's next?

Once you've connected to your data and created one (or more) data feeds, you're ready to use those data feeds in custom metrics. Learn how to build custom metrics and add them to dashboards.

Related articles