Fix caching in python3 (#806)

* caravel: fix visualization cache for python3

python3 wants bytes and not strings:

2016-07-22 10:36:09,474:INFO:root:Caching for the next 28800 seconds
2016-07-22 10:36:09,475:WARNING:root:Could not cache key 1eeb45f32960f0df0ad99a125bdaf199
2016-07-22 10:36:09,475:ERROR:root:'str' does not support the buffer interface
Traceback (most recent call last):
  File "/home/rm/caraveltest/venv/lib/python3.4/site-packages/caravel/viz.py", line 306, in get_json
    zlib.compress(self.json_dumps(payload)),
TypeError: 'str' does not support the buffer interface

Tested with memcached and pylibmc client library.

* docs: add note about using a proper memcached client library
This commit is contained in:
Riccardo Magliocchetti 2016-07-22 18:45:51 +02:00 committed by Maxime Beauchemin
parent 187149caeb
commit 8b694ddd7a
2 changed files with 13 additions and 5 deletions

View File

@ -23,7 +23,7 @@ from flask import request
from flask_babel import lazy_gettext as _
from markdown import markdown
import simplejson as json
from six import string_types
from six import string_types, PY3
from werkzeug.datastructures import ImmutableMultiDict, MultiDict
from werkzeug.urls import Href
from dateutil import relativedelta as rdelta
@ -279,7 +279,10 @@ class BaseViz(object):
if payload:
is_cached = True
try:
payload = json.loads(zlib.decompress(payload))
cached_data = zlib.decompress(payload)
if PY3:
cached_data = cached_data.decode('utf-8')
payload = json.loads(cached_data)
except Exception as e:
logging.error("Error reading cache")
payload = None
@ -302,9 +305,12 @@ class BaseViz(object):
logging.info("Caching for the next {} seconds".format(
cache_timeout))
try:
data = self.json_dumps(payload)
if PY3:
data = bytes(data, 'utf-8')
cache.set(
cache_key,
zlib.compress(self.json_dumps(payload)),
zlib.compress(data),
timeout=cache_timeout)
except Exception as e:
# cache.set call can fail if the backend is down or if

View File

@ -177,8 +177,10 @@ caching purpose. Configuring your caching backend is as easy as providing
a ``CACHE_CONFIG``, constant in your ``caravel_config.py`` that
complies with the Flask-Cache specifications.
Flask-Cache supports multiple caching backends (Redis, Memcache,
SimpleCache (in-memory), or the local filesystem).
Flask-Cache supports multiple caching backends (Redis, Memcached,
SimpleCache (in-memory), or the local filesystem). If you are going to use
Memcached please use the pylibmc client library as python-memcached does
not handle storing binary data correctly.
For setting your timeouts, this is done in the Caravel metadata and goes
up the "timeout searchpath", from your slice configuration, to your