SQL panel points a query in an included template to the same line of its parent template

[lingxiaoyang]

Tested with:

views.py

from test_debug_toolbar.models import Question

def index(request):
    return render(request, 'index.html', {'questions': Question.objects.all()})

index.html

{% extends 'base.html' %}

{% block content %}

<h5>Hello there! This is a test for debug toolbar.</h5>

<p>Below is an included template that makes a SQL query. But in debug toolbar, it is signified in "index.html", while it should be in "inner.html".</p>

{% include 'inner.html' %}

{% endblock content %}

inner.html

<p>Here's inner.html:</p>


<p>All questions:</p>
{{ questions }}

The SQL query of Question.objects.all() is executed at the line 5 of inner.html, but in the SQL panel, it is incorrectly pointed to the line 5 of its parent template index.html:

Tested with current master and develop branch, and Django 1.10.4.

For the complete test Django project: https://github.com/lingxiaoyang/test_debug_toolbar

[lingxiaoyang]

I investigated a little of this issue. The problem seems to exist only on Django 1.9 and later, most probably because of the following code (reference to utils.py):

def get_template_context(node, context, context_lines=3):
    source = getattr(node, 'source', None)
    # In Django 1.9 template Node does not have source property, Origin does
    # not reload method, so we extract contextual information from exception
    # info.
    if source:
        line, source_lines, name = get_template_source_from_source(source)
    else:
        line, source_lines, name = get_template_source_from_exception_info(
            node, context)

It is very possible that function get_template_source_from_exception_info doesn't work accurately on include template tag.

Instead, on Django 1.8 the template tracking is working correctly.

[lingxiaoyang]

Further investigation shows that Django 1.10.6 itself has the same problem when dealing with template exception, which is fixed in 1.11b1. However, Django debug toolbar gives wrong result for both versions. On an older version 1.8.11, both Django and Django debug toolbar work as expected.

Here's the comparison table:

Django version	Django	Django debug toolbar
1.8.11	Correct	Correct
1.10.6	Incorrect	Incorrect
1.11b1	Correct	Incorrect

I'll see what Django has changed in its code and perhaps submit a PR on this later.

[lingxiaoyang]

Questions to the package maintainers:

At the Django side, the bug concerning {% include %} and {% extend %} indeed exists in versions 1.9 and 1.10, and is being fixed in 1.11 LTS. However, this bugfix will not be backported to 1.9 and 1.10 (most probably because they will be out of support soon, according to Django roadmap). References: tickets 27584 and 27956.

At present, django-debug-toolbar copies the same code in Django to extract the template source. This code will need a small fix in order to work correctly in Django 1.11, but a more significant change will be required if we want it working with 1.9 and 1.10. You may compare the code of the bugfix and debug_toolbar/utils.py.

The question is therefore: do we want this fix to only work with Django 1.11 LTS and keep how it is now with versions 1.9 and 1.10; or do we also provide another solution for these two older versions?

A 2nd question: do we need a unit test on this, or is it fine to just keep it the same with Django's code?

[matthiask]

I think doing the same thing the Django maintainers do is fine; I'm alright with only being correct on 1.11 upwards.

That being said, if someone wants to do the work of supporting 1.9 and 1.10 as well despite Django itself being incorrect there I certainly won't stop them.

Unit tests are always good, and if it's possible to easily construct an unit test then it should be done (tm). If it's hard or nobody wants to invest the time then following Django is fine. If we do more than Django does by fixing the reports on 1.9 and 1.10 an unit test would clearly be a requirement though, because I expect the code involved would be more complex and we would certainly want to avoid inadvertent breakage.

[allan-simon]

i'm still seeing the issue with django-debug-toolbar 1.11 and django 1.11.23