Use Turbo Stream To Manipulate DOM Elements

Table of Contents

Django ChatGPT Tutorial Series:

  1. Introduction
  2. Create Django Project with Modern Frontend Tooling
  3. Create Chat App
  4. Partial Form Submission With Turbo Frame
  5. Use Turbo Stream To Manipulate DOM Elements
  6. Send Turbo Stream Over Websocket
  7. Using OpenAI Streaming API With Celery
  8. Use Stimulus to Better Organize Javascript Code in Django
  9. Use Stimulus to Render Markdown and Highlight Code Block
  10. Use Stimulus to Improve UX of Message Form
  11. Source Code chatgpt-django-project

In this article, we will learn what is Turbo Stream and how to use it to manipulate the DOM elements on the web page.

Objective

  1. Learn what is Turbo Stream
  2. Use Turbo Stream element to update web page in the HTTP response.

What is Turbo Stream

Turbo Streams deliver page changes as fragments of HTML wrapped in self-executing <turbo-stream> elements. Each stream element specifies an action together with a target ID to declare what should happen to the HTML inside it.

With Turbo Stream, we can use HTML snippet from web server to append, update, or remove the target DOM elements.

<turbo-stream action="append" target="messages">
  <template>
    <div id="message_1">
      This div will be appended to the element with the DOM ID "messages".
    </div>
  </template>
</turbo-stream>

<turbo-stream action="remove" target="message_1">
  <!-- The element with DOM ID "message_1" will be removed.
  The contents of this stream element are ignored. -->
</turbo-stream>

If we return HTML like this from Django server, then we have the ability to manipulate the DOM elements, without touching Javascript

Install django-turbo-response

It is tedious to write raw HTML in Django view, let's use some tool to help us.

Next, let's use django-turbo-response to help us better render Turbo Frame.

Add django-turbo-response==0.0.52 to the requirements.txt

django-turbo-response==0.0.52

(venv)$ pip install -r requirements.txt

Update hotwire_django_app/settings.py

INSTALLED_APPS = [
    'turbo_response',                # new
]


MIDDLEWARE = [
    'django.middleware.security.SecurityMiddleware',
    'turbo_response.middleware.TurboMiddleware',            # new
    ...
]

Notes:

  1. Add turbo_response to the INSTALLED_APPS
  2. Add turbo_response.middleware.TurboMiddleware to the MIDDLEWARE

Now django-turbo-response has been installed, let's use it in our project.

Use Turbo Stream To Reset Form

In the previous section, we used turbo-frame to reset the form of the page after successful form submission.

Let's build this feature using Turbo Stream.

Update chatgpt_django_app/chat/views.py

from turbo_response import TurboStream, TurboStreamResponse        # new


class MessageCreateView(CreateView):
    model = Message
    template_name = "message_create.html"
    form_class = MessageForm

    def get_success_url(self):
        return None

    def get_form_kwargs(self):
        kwargs = super().get_form_kwargs()
        kwargs["chat_pk"] = self.kwargs.get("chat_pk")
        kwargs["role"] = Message.USER
        return kwargs

    def get_empty_form(self):
        """
        Return empty form so we can reset the form
        """
        form_class = self.get_form_class()
        kwargs = self.get_form_kwargs()
        kwargs.pop("data")
        kwargs.pop("files")
        kwargs.pop("instance")
        return form_class(**kwargs)

    def form_valid(self, form):
        super().form_valid(form)
        request = self.request

        # return Turbo Stream to do partial updates on the page
        return TurboStreamResponse(
            [
                TurboStream("message-create-frame")
                    .replace.template(
                    self.template_name,
                    {
                        "form": self.get_empty_form(),
                        "request": request,
                        "view": self,
                    },
                ).response(request).rendered_content,
            ]
        )

Notes:

  1. In form_valid, we use TurboStreamResponse to return Turbo Stream to the client
  2. The replace means the turbo stream action is replace

If we launch server and submit, we can see the form is also reset after the successful submission.

If we check the network tab, the response look like this

<turbo-stream action="replace" target="message-create-frame">
  <template>
    <turbo-frame id="message-create-frame" data-turbo="true">
      <form method="post" action="/chat/1/message/create/" class="p-2 m-2">
        <input type="hidden" name="csrfmiddlewaretoken" value="hoVEov9njngrPw8ScC1qWvt1Dwz0dJvuhJ9kwgKqD6SAAyuB6HLIO1t8x41fUDxe">

        <div class="mb-3">
          <textarea name="content" cols="40" rows="10" class="rounded-lg border-gray-300 block leading-normal border px-4 text-gray-700 bg-white focus:outline-none py-2 appearance-none w-full" required id="id_content"></textarea>

          <div class="text-xs italic text-red-500"></div>
        </div>

        <button type="submit" class="transition-all bg-blue-500 hover:bg-blue-600 text-white py-2 px-4 rounded">
          Submit
        </button>
      </form>
    </turbo-frame>
  </template>
</turbo-stream>

It seems Turbo Stream can do the same thing as turbo-frame, but what is the difference?

Use Turbo Stream to Update Message List

  1. With Turbo Frame, we can do partial updates within the frame.
  2. With Turbo Stream, we can manipulate any elements on the web page

Create chatgpt_django_app/templates/message_list.html

<div id="chat-{{ chat_pk }}-message-list">
  {% for instance in object_list %}
  <div id="message_{{ instance.pk }}">
    {% if instance.role_label == 'User' %}
    <div class="p-4 m-4 max-w-full text-black rounded-lg bg-sky-100 prose" >{{ instance.content }}</div>
    {% else %}
    <div class="p-4 m-4 max-w-full bg-gray-200 rounded-lg prose">{{ instance.content }}</div>
    {% endif %}
  </div>
  {% endfor %}
</div>

Notes:

  1. To let us better reuse the HTML code, we move the message list code to a separate template
  2. And we set the id=chat-{{ chat_pk }}-message-list

Let's update chatgpt_django_app/templates/message_list_page.html

<! --- code omitted for brevity --->

<!-- Message List -->
<div class="overflow-y-auto flex-1">
  {% include 'message_list.html' with chat_pk=view.kwargs.chat_pk %}
</div>

<! --- code omitted for brevity --->

Here we use include to reuse the HTML code in message_list.html

Update chatgpt_django_app/chat/views.py

class MessageCreateView(CreateView):
    model = Message
    template_name = "message_create.html"
    form_class = MessageForm

    def get_success_url(self):
        return None

    def get_form_kwargs(self):
        kwargs = super().get_form_kwargs()
        kwargs["chat_pk"] = self.kwargs.get("chat_pk")
        kwargs["role"] = Message.USER
        return kwargs

    def get_empty_form(self):
        """
        Return empty form so we can reset the form
        """
        form_class = self.get_form_class()
        kwargs = self.get_form_kwargs()
        kwargs.pop("data")
        kwargs.pop("files")
        kwargs.pop("instance")
        return form_class(**kwargs)

    def form_valid(self, form):
        super().form_valid(form)
        request = self.request

        # return Turbo Stream to do partial updates on the page
        return TurboStreamResponse(
            [
                TurboStream("message-create-frame")
                    .replace.template(
                    self.template_name,
                    {
                        "form": self.get_empty_form(),
                        "request": request,
                        "view": self,
                    },
                ).response(request).rendered_content,
                TurboStream(f"chat-{self.kwargs['chat_pk']}-message-list")
                    .replace.template(
                    "message_list.html",
                    {
                        "chat_pk": self.kwargs['chat_pk'],
                        "object_list": Message.objects.filter(chat_id=self.kwargs['chat_pk']),
                    },
                ).response(request).rendered_content,
            ]
        )

Notes:

  1. In form_valid, we still return Turbo Stream response.
  2. This time, the single TurboStreamResponse can contain two Turbo Stream elements, actually, one response can contain multiple Turbo Stream elements.
  3. After the message is created, we use Turbo Stream to reset the form and replace the whole message list.

If we launch the server and submit in the form, we can see the form is reset after the successful submission, and the message list is also updated, without full page reload.

Note: for now, the message list can not auto scroll to the bottom to show the latest message, we will fix this later.

Use Turbo Stream to Append New Message

Let's keep moving.

It seems tedious to update the whole message list after message is created.

Let's only append the new message to the message list.

Create chatgpt_django_app/templates/message_item.html

<div id="message_{{ instance.pk }}">
  {% if instance.role_label == 'User' %}
  <div class="p-4 m-4 max-w-full text-black rounded-lg bg-sky-100 prose" >{{ instance.content }}</div>
  {% else %}
  <div class="p-4 m-4 max-w-full bg-gray-200 rounded-lg prose">{{ instance.content }}</div>
  {% endif %}
</div>

We extract message instance HTML code from message_list.html to message_item.html.

Update chatgpt_django_app/templates/message_list.html

<div id="chat-{{ chat_pk }}-message-list">
  {% for instance in object_list %}
  {% include "message_item.html" with instance=instance %}
  {% endfor %}
</div>

Update chatgpt_django_app/chat/views.py

class MessageCreateView(CreateView):
    model = Message
    template_name = "message_create.html"
    form_class = MessageForm

    def get_success_url(self):
        return None

    def get_form_kwargs(self):
        kwargs = super().get_form_kwargs()
        kwargs["chat_pk"] = self.kwargs.get("chat_pk")
        kwargs["role"] = Message.USER
        return kwargs

    def get_empty_form(self):
        """
        Return empty form so we can reset the form
        """
        form_class = self.get_form_class()
        kwargs = self.get_form_kwargs()
        kwargs.pop("data")
        kwargs.pop("files")
        kwargs.pop("instance")
        return form_class(**kwargs)

    def form_valid(self, form):
        super().form_valid(form)
        request = self.request

        # return Turbo Stream to do partial updates on the page
        return TurboStreamResponse(
            [
                TurboStream("message-create-frame")
                    .replace.template(
                    self.template_name,
                    {
                        "form": self.get_empty_form(),
                        "request": request,
                        "view": self,
                    },
                ).response(request).rendered_content,
                TurboStream(f"chat-{self.kwargs['chat_pk']}-message-list")
                    .append.template(
                    "message_item.html",
                    {
                        "instance": self.object,
                    },
                ).response(request).rendered_content,
            ]
        )

Notes:

  1. In form_valid method, we append message HTML code to the message list, instead of updating the whole message list.

If we run the server and submit in the form, we can see the form is reset after the successful submission, and the message list is updated to contain the new message.

Note: for now, the message list can not auto scroll to the bottom to show the latest message, we will fix this in later chapter.

OpenAI

Add openai to the requirements.txt

(venv)$ pip install -r requirements.txt

Update chatgpt_django_app/chat/views.py

import openai


def get_ai_response(message_pk):
    chat_instance = Message.objects.get(pk=message_pk).chat
    message_instance = Message.objects.create(
        role=Message.ASSISTANT,
        content="",
        chat=chat_instance,
    )
    messages = Message.for_openai(chat_instance.messages.all())

    try:
        response = openai.ChatCompletion.create(
            model="gpt-3.5-turbo",
            messages=messages,
            temperature=0,
        )
        message_instance.content = response['choices'][0]['message']['content']
        message_instance.save(update_fields=["content"])
    except Exception as e:
        message_instance.content = str(e)
        message_instance.save(update_fields=["content"])

    # return AI message
    return message_instance

Notes:

  1. In the get_ai_response method, we get user message instance, create a new message instance with role=Message.ASSISTANT, and use openai SDK to get AI response.
  2. As for the OpenAI API key, please set in the environment variable export OPENAI_API_KEY='sk-...' or using openai.api_key = "sk-..."

Update chatgpt_django_app/chat/views.py

class MessageCreateView(CreateView):
    model = Message
    template_name = "message_create.html"
    form_class = MessageForm

    def get_success_url(self):
        return None

    def get_form_kwargs(self):
        kwargs = super().get_form_kwargs()
        kwargs["chat_pk"] = self.kwargs.get("chat_pk")
        kwargs["role"] = Message.USER
        return kwargs

    def get_empty_form(self):
        """
        Return empty form so we can reset the form
        """
        form_class = self.get_form_class()
        kwargs = self.get_form_kwargs()
        kwargs.pop("data")
        kwargs.pop("files")
        kwargs.pop("instance")
        return form_class(**kwargs)

    def form_valid(self, form):
        super().form_valid(form)
        request = self.request

        ai_message = get_ai_response(self.object.pk)

        # return Turbo Stream to do partial updates on the page
        return TurboStreamResponse(
            [
                TurboStream("message-create-frame")
                    .replace.template(
                    self.template_name,
                    {
                        "form": self.get_empty_form(),
                        "request": request,
                        "view": self,
                    },
                ).response(request).rendered_content,
                # user message
                TurboStream(f"chat-{self.kwargs['chat_pk']}-message-list")
                    .append.template(
                    "message_item.html",
                    {
                        "instance": self.object,
                    },
                ).response(request).rendered_content,
                # AI message
                TurboStream(f"chat-{self.kwargs['chat_pk']}-message-list")
                    .append.template(
                    "message_item.html",
                    {
                        "instance": ai_message,
                    },
                ).response(request).rendered_content,
            ]
        )

Notes:

  1. In the form_valid method, we return three turbo stream elements in the turbo stream response.
  2. The first is to reset the form.
  3. The second is to append the user message to the message list.
  4. The third is to append the AI message to the message list.

You can run the code and submit a message in the form, you can see the AI response is appended to the message list.

And you will also notice, for some long AI message, you need to wait some seconds to see the full message, we will fix this in later chapter.

Conclusion

In this chapter, we learned how to use Turbo Stream to do partial updates on the page.

We do not need to write any Javascript code, just reuse server-side template to get the job done, which is awesome.

Django ChatGPT Tutorial Series:

  1. Introduction
  2. Create Django Project with Modern Frontend Tooling
  3. Create Chat App
  4. Partial Form Submission With Turbo Frame
  5. Use Turbo Stream To Manipulate DOM Elements
  6. Send Turbo Stream Over Websocket
  7. Using OpenAI Streaming API With Celery
  8. Use Stimulus to Better Organize Javascript Code in Django
  9. Use Stimulus to Render Markdown and Highlight Code Block
  10. Use Stimulus to Improve UX of Message Form
  11. Source Code chatgpt-django-project
Launch Products Faster with Django

Unlock the power of Django combined with Hotwire through SaaS Hammer. Supercharge productivity, tap into Python's rich ecosystem, and focus on perfecting your product!

Michael Yin

Michael Yin

Michael is a Full Stack Developer who loves writing code, tutorials about Django, and modern frontend techs.

He has published some tech course on testdriven.io and ebooks on leanpub.

© 2024 SaaS Hammer
Cookies Privacy