ARTICLE AD BOX
I'm rendering a chart with millions of data points using lightweight-charts + react. A task progress is displayed in the ui using a websocket. The issue arises when both are triggered simultaneously, there is a noticeable lag before progress shows up in the ui because the chart rendering is probably blocking the main thread.
Chart.jsx:
import { ColorType, createChart, CrosshairMode, LineSeries, } from "lightweight-charts"; import { useEffect, useRef } from "react"; export default function Chart({ renders }) { const chartContainerRef = useRef(); useEffect(() => { const chart = createChart(chartContainerRef.current, { layout: { background: { type: ColorType.Solid, color: "#000000" }, textColor: "#ffffff", }, width: 600, height: 300, grid: { vertLines: { visible: false, }, horzLines: { visible: false, }, }, crosshair: { mode: CrosshairMode.Magnet, }, timeScale: { backgroundColor: "#3b3b3b", timeVisible: true, rightBarStaysOnScroll: true, }, }); chart.timeScale().fitContent(); const series = chart.addSeries(LineSeries); const data = []; for (let i = 0; i < 1000000; ++i) { data.push({ time: 1766877868 + i, value: Math.floor(Math.random() * 2000), }); } series.setData(data); const observer = new ResizeObserver((entries) => { const { width, height } = entries[0].contentRect; chart.applyOptions({ width, height }); chart.timeScale().fitContent(); }); observer.observe(chartContainerRef.current); return () => { chart.remove(); observer.disconnect(); }; }, [renders]); return <div className="chart" ref={chartContainerRef}></div>; }App.jsx
import "./App.css"; import Chart from "./Chart.jsx"; import { useState } from "react"; import DirectTask from "./DirectTask.jsx"; export default function App() { const [renders, setRenders] = useState(0); return ( <div> <Chart renders={renders} /> <DirectTask setRenders={setRenders} /> </div> ); }DirectTask.jsx:
import { useEffect, useState } from "react"; import axios from "axios"; export default function DirectTask({ setRenders }) { const [progress, setProgress] = useState(""); useEffect(() => { (async () => { const socket = new WebSocket(`ws://localhost:8000/ws/example/1234`); socket.onmessage = (event) => { const message = JSON.parse(event.data); switch (message.type) { case "progress": setProgress(((message.current / message.total) * 100).toFixed(2)); } }; return () => { socket.close(); }; })(); }, []); return ( <div style={{ gap: "5px", display: "flex" }}> <button onClick={async () => { setRenders((prev) => prev + 1); await axios.get("http://localhost:8000/trigger/direct/"); }} > Direct {progress} </button> </div> ); }This is the task running on the server side:
import json from asgiref.sync import async_to_sync def exec_task(channel_layer, task_id): total = 10000 for i in range(total): async_to_sync(channel_layer.group_send)( f'task_{task_id}', { 'type': 'update_count', 'data': json.dumps( {'type': 'progress', 'current': i + 1, 'total': total} ), }, )I'm using django channels in this case, so here's consumers.py
from channels.generic.websocket import AsyncWebsocketConsumer class ExampleConsumer(AsyncWebsocketConsumer): def get_group_name(self): task_id = self.scope['path'].strip('/').split('/')[-1] return f'task_{task_id}' async def connect(self): await self.channel_layer.group_add(self.get_group_name(), self.channel_name) await self.accept() async def disconnect(self, close_code): await self.channel_layer.group_discard(self.get_group_name(), self.channel_name) async def update_count(self, event): await self.send(event['data'])routing.py
from django.urls import re_path from core.consumers import ExampleConsumer websocket_urlpatterns = [ re_path(r'ws/example/(?P<task_id>\w+)', ExampleConsumer.as_asgi()), ]The ui is shown below, it has a button which triggers the task above as well as the chart rendering. Here's how it looks with 100 data points:
3M data points (lags and starts counting abruptly from 28%):
What I tried so far and doesn't work:
Running it asynchronously leading to multiple issues: one is the inability to return a cleanup function since a promise is being returned instead, resulting in duplicated charts and two is occasionally messing up the chart on rerenders, possibly due to the same reason.
Using a web worker, which is not useful in this case since the dom manipulation is the bottleneck and it's not supported by these workers.
Note: the freeze happens due to the setData call, not just the data generation loop, so even if I get rid of the loop completely and send processed data from the server, it just improves the freeze time but doesn't prevent it.
