Speeding up SPSC Queue on C++

I started learning C++ recently and I am working on SPSC Lock-Free queue. Full code of my benchmark can be found here: GitHub. What I do, is basically pinning my Producer and Consumer threads to CPU's. With that I have avg latency around 7ms with p99 13,5ms. When I set the thread priority for each thread (on 70) like this:

inline int setThreadPriority() { sched_param param{}; param.sched_priority = 70; int policy = SCHED_FIFO; return pthread_setschedparam(pthread_self(), policy, &param); }

My latency is getting 2x worse. With AI I came up with understanding that I am facing RT Throttling and that slows my threads. If so, what are my possibilities with improving this benchmarks?
Work on isolation for my CPU ? Which is more like setting up OS itself rather than my C++ code.

I am looking for directions rather than silverbullet answer, or one line of code that will improve my program. Are those set ups (on OS level) are common in low-latency environments?

I am using Ubuntu 64, 2 sockets, 6 cores each socket, 4.10GHz Intel(R) Core(TM) i5-10600KF
1 NUMA node ( so no NUMA nodes )

-- EDIT --

Benchmark:

Total processed:1000000 Failed write:0 Min Latency: 308ns Avg Latency: 7222098ns (7.2ms) Max Latency: 13654964ns (13.6ms) p50: 7'709'850ns (7.7ms) p90: 13'434'037ns (13ms) p99: 13'623'069ns (13.6ms) p999: 13'653'354ns (13.6ms)