4b11f45f72
Currently, even though `get_sqla_engine` calls get memoized, engines are still short lived since they are attached to an models.Database ORM object. All engines created through this method have the scope of a web request. Knowing that the SQLAlchemy objects are short lived means that a related connection pool would also be short lived and mostly useless. I think it's pretty rare that connections get reused within the context of a view or Celery worker task. We've noticed on Redshift that Superset was leaving many connections opened (hundreds). This is probably due to a combination of the current process not garbage collecting connections properly, and perhaps the absence of connection timeout on the redshift side of things. This could also be related to the fact that we experience web requests timeouts (enforced by gunicorn) and that process-killing may not allow SQLAlchemy to clean up connections as they occur (which this PR may not help fixing...) For all these reasons, it seems like the right thing to do to use NullPool for external connection (but not for our connection to the metadata db!). Opening the PR for conversation. Putting this query into our staging today to run some tests. |
||
|---|---|---|
| docs | ||
| scripts | ||
| superset | ||
| tests | ||
| .codeclimate.yml | ||
| .coveralls.yml | ||
| .gitignore | ||
| .landscape.yml | ||
| .pylintrc | ||
| .travis.yml | ||
| CHANGELOG.md | ||
| CODE_OF_CONDUCT.md | ||
| CONTRIBUTING.md | ||
| ISSUE_TEMPLATE.md | ||
| LICENSE.txt | ||
| MANIFEST.in | ||
| README.md | ||
| TODO.md | ||
| alembic.ini | ||
| dev-reqs-for-docs.txt | ||
| dev-reqs.txt | ||
| dump.rdb | ||
| gen_changelog.sh | ||
| pylint-errors.sh | ||
| pypi_push.sh | ||
| run_specific_test.sh | ||
| run_tests.sh | ||
| setup.cfg | ||
| setup.py | ||
| tox.ini | ||
README.md
Superset
Apache Superset (incubating) is a modern, enterprise-ready business intelligence web application
[this project used to be named Caravel, and Panoramix in the past]
Screenshots & Gifs
View Dashboards
View/Edit a Slice
Query and Visualize with SQL Lab
Apache Superset
Apache Superset is a data exploration and visualization web application.
Superset provides:
- An intuitive interface to explore and visualize datasets, and create interactive dashboards.
- A wide array of beautiful visualizations to showcase your data.
- Easy, code-free, user flows to drill down and slice and dice the data underlying exposed dashboards. The dashboards and charts acts as a starting point for deeper analysis.
- A state of the art SQL editor/IDE exposing a rich metadata browser, and an easy workflow to create visualizations out of any result set.
- An extensible, high granularity security model allowing intricate rules on who can access which product features and datasets. Integration with major authentication backends (database, OpenID, LDAP, OAuth, REMOTE_USER, ...)
- A lightweight semantic layer, allowing to control how data sources are exposed to the user by defining dimensions and metrics
- Out of the box support for most SQL-speaking databases
- Deep integration with Druid allows for Superset to stay blazing fast while slicing and dicing large, realtime datasets
- Fast loading dashboards with configurable caching
Database Support
Superset speaks many SQL dialects through SQLAlchemy, a Python ORM that is compatible with most common databases.
Superset can be used to visualize data out of most databases:
- MySQL
- Postgres
- Vertica
- Oracle
- Microsoft SQL Server
- SQLite
- Greenplum
- Firebird
- MariaDB
- Sybase
- IBM DB2
- Exasol
- MonetDB
- Snowflake
- Redshift
- more! look for the availability of a SQLAlchemy dialect for your database to find out whether it will work with Superset
Druid!
On top of having the ability to query your relational databases, Superset has ships with deep integration with Druid (a real time distributed column-store). When querying Druid, Superset can query humongous amounts of data on top of real time dataset. Note that Superset does not require Druid in any way to function, it's simply another database backend that it can query.
Here's a description of Druid from the http://druid.io website:
Druid is an open-source analytics data store designed for business intelligence (OLAP) queries on event data. Druid provides low latency (real-time) data ingestion, flexible data exploration, and fast data aggregation. Existing Druid deployments have scaled to trillions of events and petabytes of data. Druid is best used to power analytic dashboards and applications.
Installation & Configuration
Resources
- Mailing list
- Gitter (live chat) Channel
- Docker image (community contributed)
- Slides from Strata (March 2016)
- Stackoverflow tag
- DEPRECATED Google Group
Contributing
Interested in contributing? Casual hacking? Check out Contributing.MD
Who uses Apache Superset (incubating)?
Here's a list of organizations who have taken the time to send a PR to let the world know they are using Superset. Join our growing community!
- AiHello
- Airbnb
- Amino
- Brilliant.org
- Capital Service S.A.
- Clark.de
- Digit Game Studios
- Douban
- Endress+Hauser
- FBK - ICT center
- Faasos
- GfK Data Lab
- Konfío
- Lyft
- Maieutical Labs
- Ona
- Pronto Tools
- Qunar
- Shopee
- Shopkick
- Tails.com
- Tobii
- Tooploox
- Udemy
- VIPKID
- Yahoo!
- Zalando
More screenshots
