Talkwalker Streaming API

In general, the results obtained through the Talkwalker Streaming API depend on three factors: the dataset, the connection and the rules. There are two datasets, which can be accessed:

• Global data: Contains publicly available data
• Project data: Consists of monitored and uploaded private data, which is unavailable outside of the project

The connection to the Streaming API can also be established in two ways:

• Volatile: Data is only matched while the client is connected. Disconnection results in missing data.
  • Use case: Data is immediately consumed (e.g. a live dashboard)
• Persisted: By setting up a collector instead of a direct connection, the results are buffered for 7 days. Data is available in real-time, and contrary to volatile data, it is possible for a client to disconnect and reconnect without losing any data. +
  • Use case: Handling a large amount of data and having a complete dataset. Generally recommended use case.

In the following sections, we will present 3 ways of accessing data:

• Through stream: Volatile, global data
• Through project: Volatile, project and global data
• Through collector: Persisted, global data (and project data depending on setup)

A special use case of collectors is in accessing past data, which is presented separately in the final section.

Introduction

The following table shows parameters common to all endpoints that will be used in the Talkwalker Streaming API documentation.

Credits

Each new result (independent of how many rules match) is counted as 1 credit. Updated documents come at no cost. The field appearance under matched provides this information.

Project data access

When accessing data through a Talkwalker project, either a GET or POST HTTP request with several optional parameters can be used (see Table).

Streams

Receiving results from a stream works in two steps: First, the stream is created, then the data is accessed.

Collectors

A collector allows users to define a hybrid collection of rules, based on:

Past data export through collectors

Exports allow to create asynchronous tasks that copy a selection of past data into a collector, which can then be accessed through collectors.