Preview mode doesn't work under WSGI/Gunicorn

[emorozov]

Issue Summary

Attempt to preview any page results in exception.

Steps to Reproduce

1. Open any page in the WagTail CMS for editing
2. Click Preview button

Site is running under nginx/gunicorn. When I run Django development server, this problem doesn't happen, preview works as expected. This is clearly related to gunicorn, but when I run local instance of gunicorn, it doesn't happen either. Only on the remote site.

It seems that tricks that Wagtail uses to load preview in a new window causes the problem.

After reading Wagtail source code I found this:

        preview_mode = request.GET.get('mode', page.default_preview_mode)
        response = page.serve_preview(page.dummy_request(request), preview_mode)
        response['X-Wagtail-Preview'] = 'ok'
        return response

in wagtail/wagtailadmin/views/pages.py

Apparently, differences in requests under local django devserver and gunicorn causes some malfunction in the Django authentication middleware and it fails to load currently logged in user while in preview mode.

Since this happens only in WagTail preview and all other views and dependencies functioned perfectly well in this setup for years, I assume this is a Wagtail bug.

I have "fixed" this issue in my case by changing the line from:
response = page.serve_preview(page.dummy_request(request), preview_mode)
to:
response = page.serve_preview(request, preview_mode)
On line 562 in wagtail/wagtailadmin/views/pages.py

I see that master branch uses the same dummy request technique that probably will also fail when running under gunicorn.

Technical details

AttributeError at /cms/pages/29/edit/preview/
'WSGIRequest' object has no attribute 'user'

Request Method: POST
Request URL: https://staging.royalorchard.com/cms/pages/29/edit/preview/
Django Version: 1.10.6
Python Executable: /home/rolc/.virtualenvs/dev/bin/python3
Python Version: 3.5.2
Python Path: ['/home/rolc/dev', '/home/rolc/.virtualenvs/dev/bin', '/home/rolc/.virtualenvs/dev/lib/python35.zip', '/home/rolc/.virtualenvs/dev/lib/python3.5', '/home/rolc/.virtualenvs/dev/lib/python3.5/plat-x86_64-linux-gnu', '/home/rolc/.virtualenvs/dev/lib/python3.5/lib-dynload', '/usr/lib/python3.5', '/usr/lib/python3.5/plat-x86_64-linux-gnu', '/home/rolc/.virtualenvs/dev/lib/python3.5/site-packages']
Server time: Fri, 14 Apr 2017 03:21:22 -0400
Installed Applications:
('django.contrib.admin',
 'django.contrib.auth',
 'django.contrib.contenttypes',
 'django.contrib.messages',
 'django.contrib.sessions',
 'django.contrib.sites',
 'django.contrib.humanize',
 'django.contrib.staticfiles',
 'django.contrib.admindocs',
 'django.contrib.flatpages',
 'tagging',
 'django_extensions',
 'filebrowser',
 'crispy_forms',
 'floppyforms',
 'allauth',
 'allauth.account',
 'allauth.socialaccount',
 'reversion',
 'papertrail',
 'wagtail.wagtailforms',
 'wagtail.wagtailredirects',
 'wagtail.wagtailembeds',
 'wagtail.wagtailsites',
 'wagtail.wagtailusers',
 'wagtail.wagtailsnippets',
 'wagtail.wagtaildocs',
 'wagtail.wagtailimages',
 'wagtail.wagtailsearch',
 'wagtail.wagtailadmin',
 'wagtail.wagtailcore',
 'wagtail.contrib.table_block',
 'modelcluster',
 'taggit',
 'taggit_templatetags2',
 'royal_orchard.users',
 'royal_orchard.reports',
 'royal_orchard.documents',
 'royal_orchard.emails',
 'royal_orchard.blog',
 'royal_orchard.reservations',
 'royal_orchard.vendors',
 'royal_orchard.flickr',
 'royal_orchard.search',
 'royal_orchard.userpages',
 'tinymce',
 'sorl.thumbnail',
 'constance',
 'constance.backends.database',
 'djmail',
 'bootstrap_pagination')
Installed Middleware:
('django.middleware.common.CommonMiddleware',
 'django.contrib.sessions.middleware.SessionMiddleware',
 'django.middleware.csrf.CsrfViewMiddleware',
 'django.contrib.auth.middleware.AuthenticationMiddleware',
 'django.contrib.messages.middleware.MessageMiddleware',
 'django.contrib.admindocs.middleware.XViewMiddleware',
 'django.middleware.clickjacking.XFrameOptionsMiddleware',
 'wagtail.wagtailcore.middleware.SiteMiddleware',
 'wagtail.wagtailredirects.middleware.RedirectMiddleware')


Traceback:  

File "/home/rolc/.virtualenvs/dev/lib/python3.5/site-packages/django/core/handlers/exception.py" in inner
  42.             response = get_response(request)

File "/home/rolc/.virtualenvs/dev/lib/python3.5/site-packages/django/core/handlers/base.py" in _legacy_get_response
  249.             response = self._get_response(request)

File "/home/rolc/.virtualenvs/dev/lib/python3.5/site-packages/django/core/handlers/base.py" in _get_response
  217.                 response = self.process_exception_by_middleware(e, request)

File "/home/rolc/.virtualenvs/dev/lib/python3.5/site-packages/django/core/handlers/base.py" in _get_response
  215.                 response = response.render()

File "/home/rolc/.virtualenvs/dev/lib/python3.5/site-packages/django/template/response.py" in render
  109.             self.content = self.rendered_content

File "/home/rolc/.virtualenvs/dev/lib/python3.5/site-packages/django/template/response.py" in rendered_content
  86.         content = template.render(context, self._request)

File "/home/rolc/.virtualenvs/dev/lib/python3.5/site-packages/django/template/backends/django.py" in render
  66.             return self.template.render(context)

File "/home/rolc/.virtualenvs/dev/lib/python3.5/site-packages/django/template/base.py" in render
  206.                 with context.bind_template(self):

File "/usr/lib/python3.5/contextlib.py" in __enter__
  59.             return next(self.gen)

File "/home/rolc/.virtualenvs/dev/lib/python3.5/site-packages/debug_toolbar/panels/templates/panel.py" in _request_context_bind_template
  49.         context = processor(self.request)

File "/home/rolc/dev/royal_orchard/blog/context_processors.py" in context
  10.     user = request.user

Exception Type: AttributeError at /cms/pages/29/edit/preview/
Exception Value: 'WSGIRequest' object has no attribute 'user'
Request information:
USER: Robert ***********

GET: No GET data

POST:
seo_title = ''
expire_at = ''
csrfmiddlewaretoken = '******'
body-0-type = 'paragraph'
body-count = '1'
go_live_at = ''
body-0-deleted = ''
search_description = ''
body-0-order = '0'
slug = '******-history'
title = '****** History'
next = ''
body-0-value = '<p>I feel so very privileged to be invited into your home and if we also feel a little intimidated because we are talking to folks who have generational knowledge of this wonderful house and I know as we do so we are little bit nervous about giving you advice on wh</p>'

• Python version: 3.5.2.
• Django version: 1.10.6.
• Wagtail version: 1.9.
• Browser version: Firefox 52.0.2 on Ubuntu Linux.
• Gunicorn: 19.7.1
• Nginx: 1.10.0

[alexfromvl]

I use the wagtail 1.9 on the site together with the Gunicorn. The preview_mode works well.

Probably you have a problem in the settings of the Gunicorn or Nginx.

[emorozov]

There's nothing unusual in my configuration.
Gunicorn is started with this cmdline:
/home/rolc/.virtualenvs/dev/bin/gunicorn --pid /run/gunicorn-dev/pid -b unix:/run/gunicorn-dev/socket --reload app.wsgi:application

Nginx conf is:

    location / {
        expires -1;

        proxy_read_timeout 1200;
        proxy_send_timeout 1200;
        proxy_connect_timeout 1200;

        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header Host $http_host;
        proxy_set_header X-Forwarded-SSL on;
        proxy_redirect off;

        if (!-f $request_filename) {
            proxy_pass http://unix:/run/gunicorn-dev/socket;
        }
    }

No other piece of code or third-party app fails under this configuration, only Wagtail preview. :(

[alexfromvl]

You need to look at the mistakes associated with Django. Because the error is in the user "'WSGIRequest' object has no attribute 'user'"

https://docs.djangoproject.com/en/dev/ref/middleware/#django.contrib.auth.middleware.SessionAuthenticationMiddleware

[gasman]

Looking closely at the stack trace here, I think there might be a different exception being thrown during middleware processing (probably because the dummy request is missing some other property that the middleware depends on) - as a result, it never reaches AuthenticationMiddleware, and it's failing with 'WSGIRequest' object has no attribute 'user' while attempting to render the error page.

Please can you try updating blog/context_processors.py so that it doesn't fail in the absence of request.user, and see if that gives a more helpful error?

[emorozov]

When I modify the context processor to not fail in absence of request.user, preview doesn't fail. It seems that AuthenticationMiddleware unable to load user when it's using fake request.

It seems that AuthenticationMiddleware isn't even called. I've added debug print to the AuthenticationMiddleware but it is never called when there's an error.

[emorozov]

I have added debug prints to wagtail and django and here's what I get:

Procesing request <WSGIRequest: GET '/cms/pages/29/revisions/4/view/'> with middleware <django.middleware.security.SecurityMiddleware object at 0x7fa625f1cf98>
Procesed
Procesing request <WSGIRequest: GET '/cms/pages/29/revisions/4/view/'> with middleware <django.contrib.sessions.middleware.SessionMiddleware object at 0x7fa625f1cf60>
Procesed
Procesing request <WSGIRequest: GET '/cms/pages/29/revisions/4/view/'> with middleware <django.middleware.common.CommonMiddleware object at 0x7fa625f1cb70>
Procesed
Procesing request <WSGIRequest: GET '/cms/pages/29/revisions/4/view/'> with middleware <django.contrib.auth.middleware.AuthenticationMiddleware object at 0x7fa625f1c7b8>
Procesed
Procesing request <WSGIRequest: GET '/cms/pages/29/revisions/4/view/'> with middleware <django.contrib.messages.middleware.MessageMiddleware object at 0x7fa625f1c7f0>
Procesed
Procesing request <WSGIRequest: GET '/cms/pages/29/revisions/4/view/'> with middleware <wagtail.wagtailcore.middleware.SiteMiddleware object at 0x7fa626030780>
Procesed
starting processing request <WSGIRequest: GET '/pages/info/milner-history/'>
Procesing request <WSGIRequest: GET '/pages/info/milner-history/'> with middleware <django.middleware.security.SecurityMiddleware object at 0x7fa622b92908>
Procesed
Procesing request <WSGIRequest: GET '/pages/info/milner-history/'> with middleware <django.contrib.sessions.middleware.SessionMiddleware object at 0x7fa622b92898>
Procesed
Procesing request <WSGIRequest: GET '/pages/info/milner-history/'> with middleware <django.middleware.common.CommonMiddleware object at 0x7fa622b92b00>
finished processing request <WSGIRequest: GET '/pages/info/milner-history/'>

So it seems, that all middleware is applied to the original request, but only first three entries are applied to the dummy request for some reason. There're no errors during middleware loading and application, so I still can't find the reason why AuthenticationMiddleware is never called for dummy request.

[gasman]

@emorozov Is it possible that CommonMiddleware is returning a (non-error) response? In this situation the rest of the middleware chain is skipped. (For old-style MIDDLEWARE_CLASSES definitions, we explicitly ignored those responses and continued with the chain. Perhaps we need to do that with new-style middlewares too...)

[emorozov]

@gasman Thank you! Your suggestion was right. I've found the problem: Wagtail uses its own Site model which is set to localhost by default. Since, on staging/production sites there's no localhost in settings.ALLOWED_HOSTS, CommonMiddleware was failing and exception message was silently suppressed. So, no session and authentication middleware were run, but exception was lost somewhere, so it was almost impossible to find the real cause of the problem.

I think this is still a bug. It should be either documented somewhere to set proper hostname for the wagtail Site (can it take value from Django Site which is properly set to staging hostname?) or the real exception should be displayed to the user, not silently ignored.

[BenSturmfels]

I've been able to get the preview working under Django 1.8, but not under 1.10, using the same settings (aside from rename of MIDDLEWARE/MIDDLEWARE_CLASSES). Haven't had a chance to investigate.

[BenSturmfels]

Hi @emorozov, sorry to be a pain, but could you please add some details about closing this issue?

[SalahAdDin]

@emorozov, i want to know the same.

[emorozov]

@BenSturmfels @SalahAdDin The problem only occurs when wagtail site hostname mismatches the real hostname. You need to check if you have the site hostname set correctly in the Django admin.

[BenSturmfels]

@emorozov glad you were able to resolve your issue, but if configuring the hostname incorrectly results in the somewhat baffling 'WSGIRequest' object has no attribute 'user', I think we need a better solution. Otherwise this is going to cause a headache for a lot of people unnecessarily.

[emorozov]

@BenSturmfels then the proper solution would be one of the following:

• ignore middleware errors during processing dummy wagtail request
• remove common middleware from the list of middleware when processing dummy request
• do not use middleware at all when processing fake request

Like I said in the initial message this whole dummy request thing for preview feels like a crutch. I have a feeling that it can be made in a more robust and elegant manner, but I'm not really sure I can do it.

[gasman]

Reopening this, since the current behaviour is a regression from the pre-Django-1.10 middleware where we ignored any responses (error or otherwise) originating from middleware. I'm not sure of the best way to fix it, though...

ignore middleware errors during processing dummy wagtail request

I can't see a good way to do that with Django's current middleware implementation - in the old-style middleware, we called process_request on each middleware class in turn, so we knew when a middleware was returning its own response. In the new style, the outer classes completely wrap the inner ones, so there's no way to tell from outside whether the response originates from the middleware or the actual view.

remove common middleware from the list of middleware when processing dummy request

This might work, but it feels like a "band-aid" fix - it's possible that other middlewares will have similar issues. (In fact, I'm not sure that CommonMiddleware is the culprit here - checking ALLOWED_HOSTS seems to happen in the Request object itself, so it could occur in any place that tries to retrieve hostname.)

do not use middleware at all when processing fake request

This isn't an option - the page template will often rely on variables that are inserted in middleware, such as request.user or request.site. For similar reasons I don't think we can avoid the dummy_request mechanism - the template author should be free to use any properties of the request object, so it's Wagtail's responsibility to make sure that the request object is as close as possible to the one that would be seen on a real page request.

One possibility is to add a system check that generates a warning on startup if any Site records have hostnames that aren't covered by ALLOWED_HOSTS. However, one goal I've always had with the design of Wagtail is that Site records should just work out of the box for single-site installations, and you shouldn't have to care about configuring them until you come to build a multi-site project. (For example, that's why Page.get_url() returns a local URL without the hostname on single-site installations). So maybe it's better to pre-emptively check ALLOWED_HOSTS in dummy_request, and silently fix up the host header if we can see it's going to break...

Incidentally, the unhelpfulness of the 'WSGIRequest' object has no attribute 'user' error is not Wagtail's fault - it's because you have code in context_processors.py that's not robust enough against error conditions (i.e. it's expecting to find attributes of request that aren't guaranteed to be there).

[abadger1406]

'WSGIRequest' object has no attribute 'user'

This error seen after upgraded site and try to view a form in preview mode...

(nw) dave@big5:/home/nw/nw/settings$ pip freeze
beautifulsoup4==4.6.0
certifi==2018.8.24
chardet==3.0.4
Django==2.0.8
django-debug-toolbar==1.10.1
django-extensions==2.1.2
django-modelcluster==4.2
django-taggit==0.23.0
django-treebeard==4.3
djangorestframework==3.8.2
draftjs-exporter==2.1.2
html5lib==1.0.1
idna==2.7
Pillow==5.2.0
pkg-resources==0.0.0
pytz==2018.5
requests==2.19.1
six==1.11.0
sqlparse==0.2.4
Unidecode==1.0.22
urllib3==1.23
uWSGI==2.0.17.1
wagtail==2.2.2
webencodings==0.5.1
Willow==1.1

[abadger1406]

If helpful...

(nw) dave@big5:/home/nw$ tail -f logs/uwsgi.log
File "/home/Env/nw/lib/python3.6/site-packages/wagtail/core/models.py", line 1324, in serve_preview
return self.serve(request)
File "/home/Env/nw/lib/python3.6/site-packages/wagtail/contrib/forms/models.py", line 237, in serve
form = self.get_form(page=self, user=request.user)
AttributeError: 'WSGIRequest' object has no attribute 'user'

[gasman]

@abadger1406 That one's a bit puzzling - request.user would be missing if something failed during the middleware stage, but in that case I can't see how it would proceed to the form page model's serve method. Can you post the complete traceback, please?

[jayden2]

After upgrading to 2.2.2 from 2.0 I have noticed this on production only:

AttributeError
'WSGIRequest' object has no attribute 'site'

An odd thing I've also noticed is if I change the sites port to say 200, and then refresh the preview page the page loads. Then on subsequent loads the above error happens.

[Chukwunonso]

Had this problem while upgrading to Django 2.1.7 and Wagtail 2.4 from 1.10.3 and 1.13 respectively.
Seemed none of the middleware was being triggered including the wagtail.core.middleware.SiteMiddleware which should set the site attribute to the request object. The issue went away when I changed the middleware setting from

MIDDLEWARE_CLASSES = (...)

to

MIDDLEWARE = (...)

[gasman]

See #5427 for a full diagnosis and (hopefully) fix.

[CVYY39]

Quick fix for my site was to add localhost to the ALLOWED_HOSTS string. Worked like a charm.

[tobyspark]

I had the same symptoms. I added Wagtail 2.9 to an existing Django 2.2 site, following the documentation. Everything worked, except for page previews in production, which resulted in 400 Bad Request error.

I’ve read through the various issues linked in #5427 and am none the wiser, given a) the fixes already incorporated to Wagtail 2.9 and e.g. I have none of the middleware installed, and no Wagtail site settings beyond WAGTAIL_SITE_NAME.

Adding localhost to the production ALLOWED_HOSTS results in the previews working. But clearly is a bodge. Posting as FYI; this is a fairly vanilla site, I’m surprised by this.

[gasman]

@tobyspark It sounds like you don't have the correct site hostname set under Settings -> Sites in the Wagtail admin.

Having said that, the intention behind Wagtail's site configuration is that for single-site installations, the out-of-the box setup should continue to work for as long as possible even if the hostname hasn't been filled in - so, for example, the {% pageurl %} tag doesn't include the domain part of the URL. If the only thing standing in the way of a fully-functional site in DEBUG=False mode is the mismatch between the hostname used for previewing and ALLOWED_HOSTS, then we should fix that - I think that could be done by:

• in make_preview_request, see if the intended hostname for the request is allowed by ALLOWED_HOSTS
• if not, see if the Wagtail site configuration consists of a single Site record with is_default_site=True (i.e. this Wagtail installation treats all hostnames as equivalent)
• if so, use the first item in ALLOWED_HOSTS in place of whatever the Site record's hostname says

[tobyspark]

@gasman Correct, and a slight mea-culpa in that I didn’t find those settings as I was using Wagtail when logged-in as an editor not administrator.

Nonetheless, there’s no mention that setting the site name is required in the Integrating Wagtail into a Django project documentation – and, generally, yes your intention is correct and taking the first item in ALLOWED_HOSTS seems a better approach than forcing that parameter to be set.

[SebCorbin]

@tobyspark It sounds like you don't have the correct site hostname set under Settings -> Sites in the Wagtail admin.

Having said that, the intention behind Wagtail's site configuration is that for single-site installations, the out-of-the box setup should continue to work for as long as possible even if the hostname hasn't been filled in - so, for example, the {% pageurl %} tag doesn't include the domain part of the URL. If the only thing standing in the way of a fully-functional site in DEBUG=False mode is the mismatch between the hostname used for previewing and ALLOWED_HOSTS, then we should fix that - I think that could be done by:

* in `make_preview_request`, see if the intended hostname for the request is allowed by ALLOWED_HOSTS

* if not, see if the Wagtail site configuration consists of a single Site record with `is_default_site=True` (i.e. this Wagtail installation treats all hostnames as equivalent)

* if so, use the first item in ALLOWED_HOSTS in place of whatever the Site record's hostname says

This won't work for wildcards in ALLOWED_HOSTS

[gasman]

@SebCorbin That's easily fixed - for * wildcards we're already covered by the first step, and for .example.com we just strip off the leading dot.

[henriquelalves]

I was also having this problem. My "ALLOWED_HOSTS" had no wildcards, only the hostname of the website, and I couldn't preview any page on Wagtail admin view. I'm using Caddy instead of Nginx.

What fixed it for me was changing the port of the Wagtail site config to "443" instead of the default "80". My Django application is redirecting SSL and had all SECURE_HSTS options, so maybe that was crashing the preview when using an "http" port.