Question 227421 in Docker Question posted in
The official documentation can be found here.

Docker – Superset ignoring superset_config.py settings

I’m running Superset inside a docker container. installed it thru the recommended method explained on the Superset site (clone the GIT project), then tailored the docker-compose-non-dev.yml, .env-non-dev and requirements-local.txt and superset_config.py and pulled the images… it starts and i can log into it. HOWEVER. It keeps saying that it found a default secret key

superset_worker_beat | --------------------------------------------------------------------------------
superset_worker_beat | WARNING
superset_worker_beat | --------------------------------------------------------------------------------
superset_worker_beat | A Default SECRET_KEY was detected, please use superset_config.py to override it.
superset_worker_beat | Use a strong complex alphanumeric string and use a tool to help you generate
superset_worker_beat | a sufficiently random sequence, ex: openssl rand -base64 42
superset_worker_beat | --------------------------------------------------------------------------------
superset_worker_beat | --------------------------------------------------------------------------------

It keeps saying that it will use default cache setting and they should be redefined properly to avoid data loss:

# superset_worker_beat  | Falling back to the built-in cache, that stores data in the metadata database, for the following cache: `FILTER_STATE_CACHE_CONFIG`. It is recommended to use `RedisCache`, `MemcachedCache` or another dedicated caching backend for production deployments
superset_worker_beat | 2023-07-28 12:47:06,744:WARNING:superset.utils.cache_manager:Falling back to the built-in cache, that stores data in the metadata database, for the following cache: FILTER_STATE_CACHE_CONFIG. It is recommended to use RedisCache, MemcachedCache or another dedicated caching backend for production deployments

# superset_worker_beat | Falling back to the built-in cache, that stores data in the metadata database, for the following cache: EXPLORE_FORM_DATA_CACHE_CONFIG. It is recommended to use RedisCache, MemcachedCache or another dedicated caching backend for production deployments

# superset_worker_beat | 2023-07-28 12:47:06,748:WARNING:superset.utils.cache_manager:Falling back to the built-in cache, that stores data in the metadata database, for the following cache: EXPLORE_FORM_DATA_CACHE_CONFIG. It is recommended to use RedisCache, MemcachedCache or another dedicated caching backend for production deployments

It authenticates to the default Superset page.

It keeps saying that it found a default secret key:
I’ve defined both:

       SECRET_KEY = "SuperSecretCodeIKnow"
       SUPERSET_SECRET_KEY = "SuperSecretCodeIKnow"

It keeps saying that it will use default cache setting and they should be redefined properly to avoid data loss: I’ve defined all the proper blocks acording to the SS documentation on the superset_config.py file:

       REDIS_HOST = get_env_variable("REDIS_HOST")
       REDIS_PORT = get_env_variable("REDIS_PORT")
       REDIS_CELERY_DB = get_env_variable("REDIS_CELERY_DB", "0")
       REDIS_RESULTS_DB = get_env_variable("REDIS_RESULTS_DB", "1")
       RESULTS_BACKEND = FileSystemCache("/app/superset_home/sqllab")

       CACHE_CONFIG = {
         "CACHE_TYPE": "RedisCache",
         "CACHE_DEFAULT_TIMEOUT": 300,
         "CACHE_KEY_PREFIX": "superset_",
         "CACHE_REDIS_HOST": REDIS_HOST,
         "CACHE_REDIS_PORT": REDIS_PORT,
         "CACHE_REDIS_DB": REDIS_RESULTS_DB,
       }

       DATA_CACHE_CONFIG = CACHE_CONFIG

       FILTER_STATE_CACHE_CONFIG = {
         'CACHE_TYPE': 'RedisCache',
         'CACHE_DEFAULT_TIMEOUT': 86400,
         "CACHE_REDIS_HOST": REDIS_HOST,
         "CACHE_REDIS_PORT": REDIS_PORT,
         "CACHE_REDIS_DB": 2,
         'CACHE_KEY_PREFIX': 'superset_filter_',

       EXPLORE_FORM_DATA_CACHE_CONFIG = {
         'CACHE_TYPE': 'RedisCache',
         'CACHE_DEFAULT_TIMEOUT': 86400,
         'CACHE_KEY_PREFIX': 'superset_explore_form_',
         "CACHE_REDIS_HOST": REDIS_HOST,
         "CACHE_REDIS_PORT": REDIS_PORT,
         "CACHE_REDIS_DB": 3,
         'CACHE_REDIS_URL': 'redis://redis:6379/3'
       }

It authenticates to the default Superset page.

Superset is running behind an NginX reverse proxy for the HTTPS capabilities (this works FINE).

I’ve added the proxy_fix settings to the superset_config.py:

ENABLE_PROXY_FIX = True
PROXY_FIX_CONFIG = {"x_for": 1, "x_proto": 1, "x_host": 1, "x_port": 0, "x_prefix": 1}

On this i’m trying to integrate SS with keycloak. i’ve tried the few recipes here on StackOverFlow, as well as a few other sites. Including the new flask.appbuilder capabilities for keycloak connections for authentications, by all accounts, it should be working, but all i get is always the SS login page.

my requirements-local.txt has the following packages:

itsdangerous==2.0.1 
flask-oidc==1.4.0 
Flask-OpenID==1.3.0 
flask-appbuilder==4.2.0

my client_secret.json is like this one (with the correct URLs, Realms, ID,secret):

{
  "web": {
    "issuer": "https://keycloak.local/realms/master",
    "auth_uri": "https://keycloak.local/realms/master/protocol/openid-connect/auth",
    "client_id": "myclientid",
    "client_secret": "4H6sra49h4BpV4J1SIGlx0YedwqSUNVx",
    "redirect_uris": ["https://superset.local/*"],
    "userinfo_uri": "http://keycloak:8080/realms/master/protocol/openid-connect/userinfo",
    "token_uri": "http://keycloak:8080/realms/master/protocol/openid-connect/token",
    "token_introspection_uri": "http://keycloak:8080/realms/master/protocol/openid-connect/token/introspect"
  }
}

I’ve added the following code to superset_config.py:

'''
---------------------------KEYCLOACK ----------------------------
'''
curr  =  os.path.abspath(os.getcwd())
AUTH_TYPE = AUTH_OID
OIDC_CLIENT_SECRETS =  curr + '/docker/pythonpath_dev/client_secret.json'
OIDC_ID_TOKEN_COOKIE_SECURE = False
OIDC_REQUIRE_VERIFIED_EMAIL = False
OIDC_OPENID_REALM: 'master'
OIDC_INTROSPECTION_AUTH_METHOD: 'client_secret_post'
CUSTOM_SECURITY_MANAGER = OIDCSecurityManager
AUTH_USER_REGISTRATION = True
AUTH_USER_REGISTRATION_ROLE = 'Gamma'
OIDC_VALID_ISSUERS = ['https://[url]/realms/[realm]']
'''
--------------------------------------------------------------
'''

and created a file named keycloak_security_manager.py and placed it in the /pythonpath_dev path with the following:

from flask import redirect, request
from flask_appbuilder.security.manager import AUTH_OID
from superset.security import SupersetSecurityManager
from flask_oidc import OpenIDConnect
from flask_appbuilder.security.views import AuthOIDView
from flask_login import login_user
from urllib.parse import quote
from flask_appbuilder.views import ModelView, SimpleFormView, expose
import logging
import urllib.parse

class OIDCSecurityManager(SupersetSecurityManager):

    def __init__(self, appbuilder):
        super(OIDCSecurityManager, self).__init__(appbuilder)
        if self.auth_type == AUTH_OID:
            self.oid = OpenIDConnect(self.appbuilder.get_app)
        self.authoidview = AuthOIDCView

class AuthOIDCView(AuthOIDView):

    @expose('/login/', methods=['GET', 'POST'])
    def login(self, flag=True):
        sm = self.appbuilder.sm
        oidc = sm.oid

        @self.appbuilder.sm.oid.require_login
        def handle_login():
            user = sm.auth_user_oid(oidc.user_getfield('email'))

            if user is None:
                info = oidc.user_getinfo(['preferred_username', 'given_name', 'family_name', 'email'])
                user = sm.add_user(info.get('preferred_username'), info.get('given_name'), info.get('family_name'),
                                   info.get('email'), sm.find_role('Gamma'))

            login_user(user, remember=False)
            return redirect(self.appbuilder.get_url_for_index)

        return handle_login()

    @expose('/logout/', methods=['GET', 'POST'])
    def logout(self):
        oidc = self.appbuilder.sm.oid

        oidc.logout()
        super(AuthOIDCView, self).logout()
        redirect_url = urllib.parse.quote_plus(request.url_root.strip('/') + self.appbuilder.get_url_for_login)

        return redirect(
            oidc.client_secrets.get('issuer') + '/protocol/openid-connect/logout?client_id='+ oidc.client_secrets.get('client_id')+'&post_logout_redirect_uri=' + quote(redirect_url))

On the keycloak side, i’ve created the realm, created the client, the roles inside the client, a user to whom i’ve assigned roles with the client but nothing… it ALWAYS pops the Superset auth page. I can login and use it but not thru keycloak.

Tags:

2

Answers


  1. Chosen as BEST ANSWER
    0

    I ended up starting afresh (benefits of using VMs)... I WAS using the default recommended paths and this last time went for SS 2.1.0. managed to make the SECRET_KEY, and CACHEs work, even got SS passing authentication to KC. (however now i get an error 500 after user validation). Also, was getting some gunicorn errors which were solved by adding gunincorn to the requirements-local.txt file so it gets upgraded upon start of the docker compose project.

    defined the SECRET_KEY in my .env-non-dev file in the /docker dir (created with the 'openssql rand -base64 42' command)

    and used the following as cache definitions inside my superset_config.py:

    
FILTER_STATE_CACHE_CONFIG = {
    'CACHE_TYPE': 'RedisCache',
    'CACHE_DEFAULT_TIMEOUT': 86400,
    'CACHE_KEY_PREFIX': 'superset_filter_',
    'CACHE_REDIS_URL': 'redis://localhost:6379/2'
}

EXPLORE_FORM_DATA_CACHE_CONFIG = {
    'CACHE_TYPE': 'RedisCache',
    'CACHE_DEFAULT_TIMEOUT': 86400,
    'CACHE_KEY_PREFIX': 'superset_data_explorer_',
    'CACHE_REDIS_URL': 'redis://localhost:6379/3'
}

CONTENT_SECURITY_POLICY_WARNING = False

  2. 0

    I would narrow this down to the main problem: your superset_config.py does not appear to be getting picked up at all, based on the warning about a default SECRET_KEY. Starting in version 2.1.0 Superset should refuse to start up with a default SECRET_KEY, so I assume you’re using an earlier version’s Docker image.

    Per the docs you should put your custom config settings in a file superset_config.py that sits in the directory /docker/pythonpath_dev under your base directory. The secret key config could look like:

    SECRET_KEY = 'ALongComplexStringThatYouDidntGetFromStackOverflow'

    Once you resolve that, you can tweak the cache, auth, and other settings.

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search