Skip to main content
Version: Next

Superset

Certified

Important Capabilities

CapabilityStatusNotes
Detect Deleted EntitiesOptionally enabled via stateful_ingestion
DomainsEnabled by domain config to assign domain_key
Table-Level LineageSupported by default

This plugin extracts the following:

  • Charts, dashboards, and associated metadata

See documentation for superset's /security/login at https://superset.apache.org/docs/rest-api for more details on superset's login api.

CLI based Ingestion

Install the Plugin

The superset source works out of the box with acryl-datahub.

Starter Recipe

Check out the following recipe to get started with ingestion! See below for full configuration options.

For general pointers on writing and running a recipe, see our main recipe guide.

source:
type: superset
config:
# Coordinates
connect_uri: http://localhost:8088

# Credentials
username: user
password: pass
provider: ldap

sink:
# sink configs

Config Details

Note that a . is used to denote nested fields in the YAML recipe.

FieldDescription
connect_uri
string
Superset host URL.
Default: http://localhost:8088
database_alias
map(str,string)
display_uri
string
optional URL to use in links (if connect_uri is only for ingestion)
options
object
Default: {}
password
string
Superset password.
platform_instance
string
The instance of the platform that all assets produced by this recipe belong to. This should be unique within the platform. See https://datahubproject.io/docs/platform-instances/ for more details.
provider
string
Superset provider.
Default: db
username
string
Superset username.
env
string
The environment that all assets produced by this connector belong to
Default: PROD
domain
map(str,AllowDenyPattern)
A class to store allow deny regexes
domain.key.allow
array
List of regex patterns to include in ingestion
Default: ['.*']
domain.key.allow.string
string
domain.key.ignoreCase
boolean
Whether to ignore case sensitivity during pattern matching.
Default: True
domain.key.deny
array
List of regex patterns to exclude from ingestion.
Default: []
domain.key.deny.string
string
stateful_ingestion
StatefulStaleMetadataRemovalConfig
Superset Stateful Ingestion Config.
stateful_ingestion.enabled
boolean
Whether or not to enable stateful ingest. Default: True if a pipeline_name is set and either a datahub-rest sink or datahub_api is specified, otherwise False
Default: False
stateful_ingestion.remove_stale_metadata
boolean
Soft-deletes the entities present in the last successful run but missing in the current run with stateful_ingestion enabled.
Default: True

If you were using database_alias in one of your other ingestions to rename your databases to something else based on business needs you can rename them in superset also

source:
type: superset
config:
# Coordinates
connect_uri: http://localhost:8088

# Credentials
username: user
password: pass
provider: ldap
database_alias:
example_name_1: business_name_1
example_name_2: business_name_2

sink:
# sink configs

Code Coordinates

  • Class Name: datahub.ingestion.source.superset.SupersetSource
  • Browse on GitHub

Questions

If you've got any questions on configuring ingestion for Superset, feel free to ping us on our Slack.